How to use Model API for my project?

[dleunji]

Model API : Here
My project : Ghostwriter Kant

---

Teachable NLP is a service that you can fine-tune GPT-2 models with a text file(.txt) without NLP codes.

I used my model API for my service, ’ Ghostwriter Kant '. My dream has finally come true! As a student majoring in Philosophy, it has been my wish for many years that Kant would write an essay on behalf of me. To realize it, I preprocessed the text extracted from “The Critique of Pure Reason” and trained it with the GPT-2 small/large model in Teachable NLP.

As I already explained the training process, I will go into more detail about using Model API for my own service in this post.

1. Check Model ID

After training, The Teachable NLP screen looks like below.

You can find out the model ID through the URL. In my case, URL was like below.

The model ID is a series of letters in the end of URL, that is 4fcddobl91jcdlqrouqh .

2. Model API

If you click the View API , the API guide appears. You can experience my Model API here. Let me show you more detailed guide.

The fine-tuned model is uploaded to the Servers of the url. You can get the generated text through response by sending POST Request. It is possible to test API not only using curl command as it is written in INTRO Tab, however, but also using swagger in API Tab without any codes.

Just click Try it out! , and fill out Request body for yourself. Then you can check out Response easily.

• POST

  • Just send Raw JSON Request in POST method to [Model url]/predictions/gpt-2-en-small-finetune

    e.g. https://train-qvbpkc5osu32fupvds7b-gpt2-train-teachable-ainize.endpoint.ainize.ai/predictions/gpt-2-en-small-finetune

  • The header is {'Content-Type' : 'application/json; charset=utf-8'}

  • Be careful to the type of text is array type !

  • ✰✰So you should encode the text before the Request.✰✰

  • (It is specifically guided to 3-B Backend codes using Tokenizer.)

  

  // e.g.
  {
  	"text": [464, 26231,2470], //Encoded text
    "length" : 8,
    "num_samples": 2,
  }
  

  

  Response is also array type, so you should decode the array to text.

  //e.g.
  [
  	[464, 26231, 2470, 114, 55, 4982, 99, 101, 39395, 2222],
  	[464, 26231, 2470, 39948, 57, 26553, 89, 11, 482, 56]
  ]
  

• GET

  It is used for Health Check of server.

3. Apply to my own service

It is more understandable with diagram of the API flow. I added story and design to make more interesting service, not just generating the Kantian sentences. As if Kant becomes a ghostwriter and writes an essay in place of students, I made Web page and if user let him know the beginning of the essay, he picks up where we said.

So I made the program get some input from users, encode the text, fetch the generated text using POST Request and finally decode the text from response.

A. Frontend

Get text through input of which id is 'prefix' , and if search-btn is clicked, POST request occurs through submit() .

<div class="prefix-box">
	<input type="text" id="prefix" name="prefix" placeholder="It starts with..." required>
	<div class="search-btn" type="submit" onclick="submit()">
	<!-- Search Button -->
		<i class="fas fa-search"></i>
	</div>
</div>

It is coded using Promise of Javascript.

function submit(){
	var prefix = document.getElementById("prefix");
	var prefix_value = prefix.value;
	/* If there input box is blank */
	if (prefix_value == ""){
	    alert('Please let me know how to get started!');
	    return;
	}
	var formData = new FormData();
	/* To append the prefix to form data, and make  {"context" : prefix_value}. */
	formData.append("context", prefix_value);
	/* To fetch the form to /gpt2, go through app.py, send Request, and then 
get response from app.py*/
	fetch(
	    "/gpt2", 
	    {
	        method: "POST",
	        body : formData
	    }
	)
	/* The Reponse returned from request is parameter */ 
	/* The request is encoded, and decoded like {'text: "..."} in app.py */
	.then(response => {
	    if(response.status == 200){
	        return response;
	    }
	    else if(response.status == 400){
	        throw Error("Failed1");
	    }
	    else {
	        throw Error("Failed2");
	    }
	})
	.then(response => {
	    res = response.json()
	    return res
	})
	/* To change HTML dynamically */
	.then(response => {
	    var text = response['text'] + "...<br>Now it's your turn!<br><br><br>";
	    document.getElementsByClassName("essay")[0].innerHTML = text;
	})
	/* To handle the errors */
	.catch(e => {
	    var result = document.getElementsByClassName("essay")[0];
	    result.textContent = e;
	})
}

B. Backend

Flask is used for tokenizing.

import requests
import json
from transformers import AutoTokenizer # For Encoding and Decoding 
autoTokenizer = AutoTokenizer.from_pretrained("gpt2-large")

@app.route('/gpt2', methods=['POST'])
def gpt2():
    # Extract text from form data sent from HTML
    prefix = request.form['context']
    # Encoding
    encodedText = autoTokenizer.encode(prefix)
    headers = {'Content-Type' : 'application/json; charset=utf-8'}
    data = {
        'text' : encodedText,
        'length' : 200
        # 'num_samples' key is optional, its default value is 1.
    }
    # The server URL
    url = "https://train-avgw7n5kbmsb7wrip2a8-gpt2-train-teachable-ainize.endpoint.dev.ainize.ai/predictions/gpt-2-en-small-finetune"
    # To send POST request to url, and get response
    response = requests.post(url, headers = headers, data = json.dumps(data))
    if response.status_code == 200:
        res = response.json()
	    # To Decode the value of response if response is valid
	    # 'skip_special_tokens' is optional. If it is True, special_token is excepted
        text = autoTokenizer.decode(res[0], skip_special_tokens=True)
	    # After decoding, deliver the text to frontend.
        return jsonify({'text' : text}), 200
    else:
        return jsonify({'fail' : 'error'}), response.status_code

4. Wrap up

I worried if I could create the Ghostwriter Kant in the absence of knowledge of NLP. However, through a simple API of Teachable NLP, I was able to create it with my own model. Even if you don’t know the API, other beginners can easily make service using Teachable NLP referring to my codes.

Moreover, if you upload the project with Dockerfile in Github and just fill out the URL of repository of Github in Ainize, you can serve your own AI service without cost of server.

Anyone who wants to experience my Ghostwriter Kant service, please click here!

It may be difficult for beginners to fine-tune the model and serve the project. Also the GPU needed to fine-tuning, and the cost of server is more intolerably burdensome.

But Teachable NLP, Ainize solved the problems. Just write some codes and clicks, then you can complete your own project! How about boasting of your service in this forum too!