Skip to content

Instantly share code, notes, and snippets.

@HeenaR17

HeenaR17/preproc.py

Created December 21, 2020 17:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save HeenaR17/878420c31ef41d627adbd49eaaa7a772 to your computer and use it in GitHub Desktop.
Save HeenaR17/878420c31ef41d627adbd49eaaa7a772 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
preproc.py
files.upload()
pd.set_option('display.max_colwidth', None)
df = pd.read_csv('train.tsv',sep='\t')
df.head(5)
df.describe()
paraphrase_train_df = df[df['label']==1]
paraphrase_train_df.head(5)
paraphrase_train_df.describe()
paraphrase_train_df["prefix"] = "generate paraphrase"
paraphrase_train_df = paraphrase_train_df.rename(columns={"sentence1": "input_text", "sentence2": "target_text"})
df = pd.read_csv('dev.tsv',sep='\t')
paraphrase_dev_df = df[df['label']==1]
paraphrase_dev_df["prefix"] = "generate paraphrase"
paraphrase_dev_df = paraphrase_dev_df.rename(columns={"sentence1": "input_text", "sentence2": "target_text"})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment