A paper summarizer with Python and GPT-3

mediumThis post was originally published by Lucas Soares at Medium [AI]

Photo by Aleks Dorohovich on Unsplash

Using python and openai’s GPT-3 to summarize AI papers

When access to the GPT-3 api was released to researchers and engineers (upon request) I immediately requested it so I could see what kind of interesting tools one could write and what kind of interesting research questions could be asked.

Photo by Seven Shooter on Unsplash

Steps to summarize a paper with GPT-3

1. Download the paper

import openai
import wget
import pathlib
import pdfplumber
import numpy as np
pip install openai
pip install wget
pip install pdfplumber
def getPaper(paper_url, filename="random_paper.pdf"):
    """
    Downloads a paper from it's arxiv page and returns
    the local path to that file.
    """
    downloadedPaper = wget.download(paper_url, filename)    
    downloadedPaperFilePath = pathlib.Path(downloadedPaper)

return downloadedPaperFilePath

2. Convert from pdf to text

paperFilePath = "random_paper.pdf"
paperContent = pdfplumber.open(paperFilePath).pages

def displayPaperContent(paperContent, page_start=0, page_end=5):
for page in paperContent[page_start:page_end]:
print(page.extract_text())
displayPaperContent(paperContent)

# Output 
Understanding training and generalization in deeplearning by Fourier analysisZhi-QinJohnXu∗8NewYorkUniversityAbuDhabi1AbuDhabi129188,UnitedArabEmirates0zhiqinxu@nyu.edu2 voN Abstract 92 Background: It is still an open research area to theoretically understand why  DeepNeuralNetworks(DNNs)—equippedwithmanymoreparametersthantrain- ] ing data and trained by (stochastic) gradient-based methods—often achieve re-Gmarkably low generalization error. Contribution: We study DNN training byL Fourier analysis. Our theoretical frameworkexplains: i) DNN with (stochastic). gradient-based methods often endows low-frequency components of the targetsc function with a higher priority during the training; ii) Small initialization leads[ to good generalizationability of DNN while preservingthe DNN’s ability to fit   any function. These results are further confirmed by experiments of DNNs
...
...

3. Feed the text to the GPT-3 model using the openai api

def showPaperSummary(paperContent):
    tldr_tag = "n tl;dr:"
    openai.organization = 'organization key'
    openai.api_key = "your api key"
    engine_list = openai.Engine.list() # calling the engines available from the openai api

for page in paperContent:
text = page.extract_text() + tldr_tag
response = openai.Completion.create(engine=”davinci”,prompt=text,temperature=0.3,
max_tokens=140,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=[“n”]
)
print(response[“choices”][0][“text”])

tldr_tag = "n tl;dr:"
openai.organization = "the openai organization key"
openai.api_key = "your api key"
engine_list = openai.Engine.list() # calling the engines available  # from the openai api
for page in paperContent:    
        text = page.extract_text() + tldr_tag
response = openai.Completion.create(engine="davinci",prompt=text,temperature=0.3,
            max_tokens=140,
            top_p=1,
            frequency_penalty=0,
            presence_penalty=0,
            stop=["n"]
        )
        print(response["choices"][0]["text"])

4. Show the summary

paperContent = pdfplumber.open(paperFilePath).pages
showPaperSummary(paperContent)# Output:The power spectrum of the tanh function exponentially decays w.r.t. frequency.

Thegradient-basedtrainingwiththeDNNwithonehiddenlayerwithtanhfunction
ThetotallossfunctionofasinglehiddenlayerDNNis

The initial weights of a deep neural network are more important than the initial biases.
We can use Fourier analysis to understand the gradient-based optimization of DNNs on real data and pure noise. We found that DNNs have a better generalization ability when the loss function is flatter. We also found that the DNN generalization ability is better when the DNN parameters are smaller.

The code is available at https://github.com/yuexin-wang/DeepLearning-Music .

Theorem 3 is true.
We can use the same method to prove the theorem.
The DNN is a very powerful tool that can be used for many things. It is not a panacea, but it is a very powerful tool.
Deep learning is only one of many tools for building intelligent systems.

Putting it all together

import openai
import wget
import pathlib
import pdfplumber
import numpy as np

def getPaper(paper_url, filename=”random_paper.pdf”):
“””
Downloads a paper from it’s arxiv page and returns
the local path to that file.
“””
downloadedPaper = wget.download(paper_url, filename)
downloadedPaperFilePath = pathlib.Path(downloadedPaper)

return downloadedPaperFilePath

def showPaperSummary(paperContent):
tldr_tag = “n tl;dr:”
openai.organization = ‘org-cur38Rg6pd6JFV2pDFNAFtEh’
openai.api_key = “sk-JXm3aEq3AAUezUoKjGD2qCJ46xf5aAEUmMmUsbwf”
engine_list = openai.Engine.list()

for page in paperContent:
text = page.extract_text() + tldr_tag
response = openai.Completion.create(engine=”davinci”,prompt=text,temperature=0.3,
max_tokens=140,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=[“n”]
)
print(response[“choices”][0][“text”])

paperContent = pdfplumber.open(paperFilePath).pages
showPaperSummary(paperContent)

Thoughts on Summarization

Spread the word

This post was originally published by Lucas Soares at Medium [AI]

Related posts