Added OpenAICompletionApp example, along with some useful models

README.md

@@ -1,6 +1,6 @@
 # ZIO Restful webservice example
 
-Using zio 2.0, zio-http, zio-json, quill, H2, twirl, zio-logging, zio-cache, zio-actors, zio-spark
+Using zio 2.0, zio-http, zio-json, quill, H2, twirl, zio-logging, zio-cache, zio-actors, zio-spark, openai
 
 Now targeting zio-http 0.0.3
 
@@ -91,6 +91,13 @@ Now targeting zio-http 0.0.3
 - http://localhost:8080/spark/job
 - http://localhost:8080/spark/wordcount
 
+## OpenAICompletionApp
+(Modelled after https://github.com/openai/openai-quickstart-python.git)
+
+To use it you'll have to fill in your OpenAI `OPENAI_API_KEY`
+
+- http://localhost:8080/openai/superhero/Dog
+
 ## To run
 
 ```scala

src/main/scala/com/felstar/restfulzio/MainApp.scala

@@ -9,6 +9,7 @@ import com.felstar.restfulzio.helloworld.HelloWorldApp
 import com.felstar.restfulzio.noenv.NoEnvApp
 import com.felstar.restfulzio.staticserver.StaticApp
 import com.felstar.restfulzio.hellotwirl.HelloTwirlApp
+import com.felstar.restfulzio.openai.OpenAICompletionApp
 import com.felstar.restfulzio.spark.SparkApp
 import com.felstar.restfulzio.stream.StreamApp
 import com.felstar.restfulzio.videos.{InmemoryVideoRepo, PersistentVideoRepo, VideoApp}
@@ -64,7 +65,7 @@ object MainApp extends ZIOAppDefault {
     _ <- ZIO.logInfo(args.toString).provide(logger)
     serverFibre <- Server.serve((NoEnvApp()  @@ requestMiddleWare ++ HelloWorldApp() ++ DownloadApp() ++
       CounterApp() ++ VideoApp() ++ ActorsApp() ++ HelloTwirlApp() ++ (if (enableSpark) SparkApp() else Http.empty)  ++
-      DelayApp() ++ StreamApp() ++ ClientApp() ++ StaticApp()) @@ middlewares)
+      DelayApp() ++ StreamApp() ++ ClientApp() ++ OpenAICompletionApp() ++ StaticApp()) @@ middlewares)
       .provide(
         Server.default,
         Scope.default,

src/main/scala/com/felstar/restfulzio/openai/CreateCompletionRequest.scala

@@ -0,0 +1,45 @@
+package com.felstar.restfulzio.openai
+
+import zio.json._
+/**
+ * @param model ID of the model to use. You can use the [List models](/docs/api-reference/models/list) API to see all of your available models, or see our [Model overview](/docs/models/overview) for descriptions of them.
+ * @param prompt The prompt(s) to generate completions for, encoded as a string, array of strings, array of tokens, or array of token arrays.  Note that <|endoftext|> is the document separator that the model sees during training, so if a prompt is not specified the model will generate as if from the beginning of a new document. 
+ * @param suffix The suffix that comes after a completion of inserted text. for example: '''test.'''
+ * @param max_tokens The maximum number of [tokens](/tokenizer) to generate in the completion.  The token count of your prompt plus `max_tokens` cannot exceed the model's context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096).  for example: '''16'''
+ * @param temperature What [sampling temperature](https://towardsdatascience.com/how-to-sample-from-language-models-682bceb97277) to use. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer.  We generally recommend altering this or `top_p` but not both.  for example: '''1'''
+ * @param top_p An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.  We generally recommend altering this or `temperature` but not both.  for example: '''1'''
+ * @param n How many completions to generate for each prompt.  **Note:** Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens` and `stop`.  for example: '''1'''
+ * @param stream Whether to stream back partial progress. If set, tokens will be sent as data-only [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Event_stream_format) as they become available, with the stream terminated by a `data: [DONE]` message. 
+ * @param logprobs Include the log probabilities on the `logprobs` most likely tokens, as well the chosen tokens. For example, if `logprobs` is 5, the API will return a list of the 5 most likely tokens. The API will always return the `logprob` of the sampled token, so there may be up to `logprobs+1` elements in the response.  The maximum value for `logprobs` is 5. If you need more than this, please contact us through our [Help center](https://help.openai.com) and describe your use case. 
+ * @param echo Echo back the prompt in addition to the completion 
+ * @param stop Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence. 
+ * @param presence_penalty Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.  [See more information about frequency and presence penalties.](/docs/api-reference/parameter-details) 
+ * @param frequency_penalty Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.  [See more information about frequency and presence penalties.](/docs/api-reference/parameter-details) 
+ * @param best_of Generates `best_of` completions server-side and returns the \"best\" (the one with the highest log probability per token). Results cannot be streamed.  When used with `n`, `best_of` controls the number of candidate completions and `n` specifies how many to return – `best_of` must be greater than `n`.  **Note:** Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens` and `stop`. 
+ * @param logit_bias Modify the likelihood of specified tokens appearing in the completion.  Accepts a json object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this [tokenizer tool](/tokenizer?view=bpe) (which works for both GPT-2 and GPT-3) to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.  As an example, you can pass `{\"50256\": -100}` to prevent the <|endoftext|> token from being generated. 
+ * @param user A unique identifier representing your end-user, which will help OpenAI to monitor and detect abuse. [Learn more](/docs/usage-policies/end-user-ids).  for example: '''user-1234'''
+ */
+case class CreateCompletionRequest (
+  model: String,
+  prompt: Option[String] = None,
+  suffix: Option[String] = None,
+  max_tokens: Option[Int] = None,
+  temperature: Option[BigDecimal] = None,
+  top_p: Option[BigDecimal] = None,
+  n: Option[Int] = None,
+  stream: Option[Boolean] = None,
+  logprobs: Option[Int] = None,
+  echo: Option[Boolean] = None,
+  stop: Option[List[String]] = None,
+  presence_penalty: Option[BigDecimal] = None,
+  frequency_penalty: Option[BigDecimal] = None,
+  best_of: Option[Int] = None,
+  logit_bias: Option[Map[String, Int]] = None,
+  user: Option[String] = None
+)
+
+object CreateCompletionRequest {
+  implicit val encoder: JsonEncoder[CreateCompletionRequest] = DeriveJsonEncoder.gen[CreateCompletionRequest]
+  implicit val decoder: JsonDecoder[CreateCompletionRequest] = DeriveJsonDecoder.gen[CreateCompletionRequest]
+}
+

src/main/scala/com/felstar/restfulzio/openai/CreateCompletionResponse.scala

@@ -0,0 +1,25 @@
+package com.felstar.restfulzio.openai
+
+import zio.json.{DeriveJsonDecoder, DeriveJsonEncoder, JsonDecoder, JsonEncoder}
+
+/**
+ * @param id 
+ * @param `object` 
+ * @param created 
+ * @param model 
+ * @param choices 
+ * @param usage 
+ */
+case class CreateCompletionResponse (
+  id: String,
+  `object`: String,
+  created: Int,
+  model: String,
+  choices: List[CreateCompletionResponseChoices],
+  usage: Option[CreateCompletionResponseUsage] = None
+)
+
+object CreateCompletionResponse {
+  implicit val encoder: JsonEncoder[CreateCompletionResponse] = DeriveJsonEncoder.gen[CreateCompletionResponse]
+  implicit val decoder: JsonDecoder[CreateCompletionResponse] = DeriveJsonDecoder.gen[CreateCompletionResponse]
+}

src/main/scala/com/felstar/restfulzio/openai/CreateCompletionResponseChoices.scala

@@ -0,0 +1,21 @@
+package com.felstar.restfulzio.openai
+
+import zio.json.{DeriveJsonDecoder, DeriveJsonEncoder, JsonDecoder, JsonEncoder}
+
+/**
+ * @param text 
+ * @param index 
+ * @param logprobs 
+ * @param finish_reason 
+ */
+case class CreateCompletionResponseChoices (
+  text: Option[String] = None,
+  index: Option[Int] = None,
+  logprobs: Option[CreateCompletionResponseLogprobs] = None,
+  finish_reason: Option[String] = None
+)
+
+object CreateCompletionResponseChoices {
+  implicit val encoder: JsonEncoder[CreateCompletionResponseChoices] = DeriveJsonEncoder.gen[CreateCompletionResponseChoices]
+  implicit val decoder: JsonDecoder[CreateCompletionResponseChoices] = DeriveJsonDecoder.gen[CreateCompletionResponseChoices]
+}

src/main/scala/com/felstar/restfulzio/openai/CreateCompletionResponseLogprobs.scala

@@ -0,0 +1,20 @@
+package com.felstar.restfulzio.openai
+import zio.json.{DeriveJsonDecoder, DeriveJsonEncoder, JsonDecoder, JsonEncoder}
+
+/**
+ * @param tokens 
+ * @param token_logprobs 
+ * @param top_logprobs 
+ * @param text_offset 
+ */
+case class CreateCompletionResponseLogprobs (
+  tokens: Option[List[String]] = None,
+  token_logprobs: Option[List[BigDecimal]] = None,
+  top_logprobs: Option[List[Map[String, Double]]] = None,
+  text_offset: Option[List[Int]] = None
+)
+
+object CreateCompletionResponseLogprobs {
+  implicit val encoder: JsonEncoder[CreateCompletionResponseLogprobs] = DeriveJsonEncoder.gen[CreateCompletionResponseLogprobs]
+  implicit val decoder: JsonDecoder[CreateCompletionResponseLogprobs] = DeriveJsonDecoder.gen[CreateCompletionResponseLogprobs]
+}

src/main/scala/com/felstar/restfulzio/openai/CreateCompletionResponseUsage.scala

@@ -0,0 +1,18 @@
+package com.felstar.restfulzio.openai
+import zio.json.{DeriveJsonDecoder, DeriveJsonEncoder, JsonDecoder, JsonEncoder}
+
+/**
+ * @param prompt_tokens
+ * @param completion_tokens
+ * @param total_tokens
+ */
+case class CreateCompletionResponseUsage (
+  prompt_tokens: Int,
+  completion_tokens: Int,
+  total_tokens: Int
+)
+
+object CreateCompletionResponseUsage {
+  implicit val encoder: JsonEncoder[CreateCompletionResponseUsage] = DeriveJsonEncoder.gen[CreateCompletionResponseUsage]
+  implicit val decoder: JsonDecoder[CreateCompletionResponseUsage] = DeriveJsonDecoder.gen[CreateCompletionResponseUsage]
+}

src/main/scala/com/felstar/restfulzio/openai/OpenAICompletionApp.scala

@@ -0,0 +1,71 @@
+package com.felstar.restfulzio.openai
+
+import zio.json._
+
+import zio.{ZIO}
+import zio.http._
+import zio.http.model.{Header, Headers, Method}
+import zio._
+
+// Modelled after the quickstart openai completion app https://github.com/openai/openai-quickstart-python.git
+
+/** An http app that:
+  *   - Accepts a `Request` and returns a `Response`
+  *   - Could fail
+  *   - Uses EventLoopGroup with ChannelFactory Env for Client
+  */
+object OpenAICompletionApp {
+
+  val HOST = "https://api.openai.com/v1"
+
+  def apply(): Http[
+    Client,
+    Throwable,
+    Request,
+    Response
+  ] =
+    Http.collectZIO[Request] {
+      case Method.GET -> !! / "openai" / "superhero" / animal =>
+
+        val url = s"$HOST/completions"
+        val prompt=
+          s"""Suggest three names for an animal that is a superhero.
+
+        Animal: Cat
+        Names: Captain Sharpclaw, Agent Fluffball, The Incredible Feline
+        Animal: Dog
+        Names: Ruff the Protector, Wonder Canine, Sir Barks-a-Lot
+        Animal: ${animal.toUpperCase}
+        Names:"""
+        val json = for {
+          open_api_key_property <- System.property("OPENAI_API_KEY")
+          open_api_key_env <- System.env("OPENAI_API_KEY")
+          open_api_key = open_api_key_property.getOrElse(open_api_key_env, "ENTER_OPENAI_API_KEY")
+          res <- Client.request(
+            url,
+            method = Method.POST,
+            headers =
+              Headers(Header("Content-type", "application/json; charset=UTF-8"), Header("Authorization", s"Bearer $open_api_key")),
+            content = Body.fromString(
+              CreateCompletionRequest(model="text-davinci-003", prompt= Some(prompt), temperature = Some(0.6)).toJson)
+          )
+          _ <- ZIO.logInfo(s"Called $url")
+          string <- res.body.asString
+          _ <- ZIO.logInfo(string)
+          response = string.fromJson[CreateCompletionResponse].toOption
+          json = response.map(_.toJsonPretty).getOrElse(string)
+          // It can return an Error, especially if no valid OPEN_API_KEY
+          // so we'd like to expose this to the user
+          // e.g. "error": {
+          //        "message": "Incorrect API key provided: ENTER_OP********_KEY. You can find your API key at https://beta.openai.com.",
+          //        "type": "invalid_request_error",
+          //        "param": null,
+          //        "code": "invalid_api_key"
+          //    }
+        } yield json
+        json.map(Response.json(_))
+
+
+    }
+
+}