PaLM-2 (chat-bison) API is now generally available

Introduction

In the fast-paced world of artificial intelligence, Google continues to push the boundaries of innovation with the release of PaLM-2 (Pathway Language Model) in their Generative AI Studio. This release marks a significant milestone in the development of chat applications, enabling developers to leverage advanced generative AI capabilities. In this article, we will explore the features and implications of Google PaLM-2 for chat applications.

PaLM-2, the latest iteration of the Pattern-Attributed Language Model, represents a breakthrough in chat-based AI technology. It is designed to understand and generate human-like responses in natural language conversations. Powered by deep learning algorithms and trained on vast amounts of text data, PaLM-2 demonstrates remarkable fluency and contextual understanding, making it an invaluable tool for chat application developers.

Google’s PaLM APIs for chat (chat-bison) provide developers with a powerful interface to integrate PaLM-2 into their chat applications. These APIs offer a range of functionalities, including sentiment analysis, language translation, and response generation. By harnessing the power of PaLM-2, developers can create chat applications that deliver more natural, contextually aware responses, thereby enhancing user experiences.

Integration and Usage

You can start use PaLM-2 in Vertex AI, Generative AI Studio like this:

Developers can interact with the model in a similar way to ChatGPT, initiating conversations and receiving responses:

API access is also available, allowing developers to integrate PaLM-2 into their applications. For example, in Node.JS, API access can be easily implemented by providing the filename of the Google Service account JSON file obtained from the Cloud Console:

import { JWT } from "google-auth-library";

const API_ENDPOINT = "us-central1-aiplatform.googleapis.com";
const URL = `https://${API_ENDPOINT}/v1/projects/${process.env.GOOGLE_KEY}/locations/us-central1/publishers/google/models/chat-bison@001:predict`;

const getIdToken = async () => {
    const client = new JWT({
        keyFile: "./google.json",
        scopes: ["https://www.googleapis.com/auth/cloud-platform"],
    });
    const idToken = await client.authorize();
    return idToken.access_token;
};

export const getTextPalm = async (prompt, temperature) => {
    const headers = {
        Authorization: `Bearer ` + (await getIdToken()),
        "Content-Type": "application/json",
    };

    const data = {
        instances: [
            {
                context: "",
                examples: [],
                messages: [
                    {
                        author: "user",
                        content: prompt,
                    },
                ],
            },
        ],
        parameters: {
            temperature: temperature || 0.5,
            maxOutputTokens: 1024,
            topP: 0.8,
            topK: 40,
        },
    };

    const response = await fetch(URL, {
        method: "POST",
        headers,
        body: JSON.stringify(data),
    });

    if (!response.ok) {
        console.error(response.statusText);
        throw new Error("Request failed " + response.statusText);
    }

    const result = await response.json();
    return result.predictions[0].candidates[0].content;
};

Currently, PaLM-2 responds only in English, but support for other languages is expected to be added in the near future. The cost of using the PaLM-2 API is relatively affordable at $0.0005 per 1K characters. However, it’s worth noting that Google is currently offering a full discount on the usage fees.

Testing and Impressive Results:

During testing, PaLM-2 demonstrated impressive capabilities, particularly in storytelling. The generated stories produced by PaLM-2 are comparable to results achieved by models like GPT-4.