Gen AI is the hottest buzzword in the industry at the moment, with companies all trying to gain an edge. While AI is not a relatively new technology, only recently has it it become easy for technicians of all skill levels to explore its capabilities. One of the organizations making this possible is AWS of course. The AWS homepage is mostly AI/ML content so it’s clear what they deem important and where their focus is. With numerous offerings and products, the one that seems to gives the quickest pathway to explore GenAI (generative artificial intelligence) is Bedrock. It’s a managed, serverless service that exposes foundational models (FM) from an API.
As a simple DevOps/Platform engineer, my skillset up until this point has not been in machine learning. I’m sure I can get up to speed, in time, and learn the vast array of foundational skills to be effective in the domain but I have applications and platforms to build TODAY. In order to be successful in this field platform teams need to innovate rapidly. This is where Bedrock comes in, developers that are familiar with REST APIs are able to rapidly integrate Gen AI into their applications from a familiar workflow. Making GET requests are fundamental for most developers and what makes Bedrock so powerful. With a single request, I’m able to interface with FM’s and build Gen AI applications. Bedrock goes even farther with its ability to further train these FM’s using custom datasets in order to extend a models capability with information that’s important to your organization or use case.
This purpose of this post isn’t to further train a FM, but as a demo for Bedrock. A proof of concept if you will. I’m going to demonstrate just how simple it is to create a chatbot, and you know I’m going to do it in the terminal with our favorite Golang package Bubbletea! Using mostly code from Bedrock and Charm samples, we’ll end up with a fully functioning chatbot that’s ready for immediate use.
Prerequisites
To complete this tutorial, you'll need the following:
- An AWS account with privileges to request access to models in the us-east-1 region
- Golang installed (used 1.22.2)
- Internet connection
- A general understanding of Golang and the Bubbletea package
- I'm using a Macbok to build/run this code (Unix based OS)
Request Access to Foundation Models
Bedrock does not come ready to go out the gate. GenAI takes a level of responsibility and AWS doesn’t make the service openly available for use, just having an AWS account isn’t enough. You must first request access to the FMs you want to interact with. In order to do that complete the following instructions, they come from the AWS guide here:
- Log into your AWS account, ensure you are in the us-east-1 region
- Search for the Bedrock service
- On the left menu scroll to Bedrock configurations and select Model access
- This guide is written for interactions with FM Titan Text G1 - Express so locate it in the list of models. You can also use one of the other Titan Text models such as:
- Titan Text G1 - Lite
- Titan Text G1 - Premier
FMTitan Text G1 at the time of writing is available in the us-east-1 region. If you aren't the FM please ensure you're in that region.
- Select the Available to request button
- You’ll be taken to another menu where you can select multiple models
- Select all the ones you want, then progress through menus until you can Submit
Approval for my account was instant but I’ve read that it can be hours or days depending on varying factors.
After approval, let's get into some code. The demo can be cloned here, we'll walk through some of the functions for a better understanding of what's happening then run it.
Most of this code comes from the AWS and Bubbletea sample codes respectively.
titanModel.go
This file holds the code that's going to complete our request to Bedrock. We're using packages from the AWS SDK for Go in order to communicate with the Bedrock API.
package titan
import (
"context"
"encoding/json"
"log"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"
)
// InvokeModelWrapper is a wrapper that holds a reference to the Bedrock runtime client
type InvokeModelWrapper struct {
BedrockRuntimeClient *bedrockruntime.Client
}
// TitanTextRequest represents the request payload for invoking the Titan text generation model
type TitanTextRequest struct {
InputText string `json:"inputText"`
TextGenerationConfig TextGenerationConfig `json:"textGenerationConfig"`
}
// TextGenerationConfig represents the configuration settings for text generation
type TextGenerationConfig struct {
Temperature float64 `json:"temperature"`
TopP float64 `json:"topP"`
MaxTokenCount int `json:"maxTokenCount"`
StopSequences []string `json:"stopSequences,omitempty"`
}
// TitanTextResponse represents the response from the Titan text generation model
type TitanTextResponse struct {
InputTextTokenCount int `json:"inputTextTokenCount"`
Results []Result `json:"results"`
}
// Result represents a single result from the Titan text generation model
type Result struct {
TokenCount int `json:"tokenCount"`
OutputText string `json:"outputText"`
CompletionReason string `json:"completionReason"`
}
- package: We put this file in a different package from main as we don't want to have a main() function here. We'll import this file in main.go in the next step
- import: Import the packages we need, note the ones from AWS.
- The remaining structs are used to structure our JSON body, the object will then get passed to Bedrock to structure what the response should include. More on working with JSON in Golang can be found here.
You may be asking, "how do you know what data should be in the response and request"? The answer is in the documentation, link here.
// InvokeTitanText invokes the Titan text generation model with the given prompt
func (wrapper InvokeModelWrapper) InvokeTitanText(ctx context.Context, prompt string) (*bedrockruntime.InvokeModelOutput, error) {
modelId := "amazon.titan-text-express-v1"
// Create the request payload
body, err := json.Marshal(TitanTextRequest{
InputText: prompt,
TextGenerationConfig: TextGenerationConfig{
Temperature: 0,
TopP: 1,
MaxTokenCount: 3000,
},
})
if err != nil {
log.Fatal("failed to marshal", err)
}
// Invoke the model passing the body and model ID
output, err := wrapper.BedrockRuntimeClient.InvokeModel(ctx, &bedrockruntime.InvokeModelInput{
ModelId: aws.String(modelId),
ContentType: aws.String("application/json"),
Body: body,
})
if err != nil {
ProcessError(err, modelId)
}
return output, nil
}
InvokeTitanText() prepares our communication and responses with Bedrock, taking configurations for the Titan model response, accepting a context and prompt (or request) to send to Titan.
- Temperature: controls the word randomness for the Titan models responses, smaller numbers for less randomness.
- TopP: controls probability for token selection, smaller numbers for less diversity in response.
- MaxTokenCount: controls the maximum number of tokens to use in a response with 4096 being the default.
We then call the InvokeModel function from our wrapper BedrockRuntimeClient passing our modelId string variable. Notice we're specifying that I want the contents to be JSON, and the model configurations including the prompt (we'll later get with Bubbletea) along with the response configurations we discussed previously. If you wanted to use a different FM change the modelId variable to reflect the change.
// ProcessOutput processes the output from the Titan model
func ProcessOutput(userPrompt string) (string, error) {
// Load the AWS configuration
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))
if err != nil {
log.Fatalf("failed to load configuration, %v", err)
}
// Create a new Bedrock runtime client
client := bedrockruntime.NewFromConfig(cfg)
wrapper := InvokeModelWrapper{
BedrockRuntimeClient: client,
}
ctx := context.Background()
var response TitanTextResponse
// Invoke the Titan text generation model
output, err := wrapper.InvokeTitanText(ctx, userPrompt)
if err := json.Unmarshal(output.Body, &response); err != nil {
log.Fatal("failed to unmarshal", err)
}
if err != nil {
log.Fatalf("failed to invoke TitanText model, %v", err)
}
return response.Results[0].OutputText, nil
}
We call InvokeTitanText() in this function, ProcessOutput(). Here we're configuring our BedrockRuntime client declaring we want to work in the us-east-1 region and passing that to our wrapper for use later. We instantiate the background context then call InvokeTitanText() to unmarshal our JSON response with its return output value.
Finally, we dig down to the OutputText as our return for later use in the main.go file where our Bubbletea stuff is happening.
main.go
Our Bubbletea tutorial can be found here and from the Bubbletea sample code above so I'm not going to walk through it as closely, but I did want to touch on the custom Bubbletea tea.Msg I'm using.
...
// Define a message type for the response from the Titan model
type responseMsg struct {
response string
err error
}
...
// Command to process the output from the Titan model
func processOutputCmd(userMessage string) tea.Cmd {
return func() tea.Msg {
response, err := titan.ProcessOutput(userMessage)
return responseMsg{response: response, err: err}
}
}
...
case tea.KeyCtrlC, tea.KeyEsc:
// Quit the program on Ctrl+C or Esc
fmt.Println(m.textarea.Value())
return m, tea.Quit
case tea.KeyEnter:
userMessage := m.textarea.Value()
m.messages = append(m.messages, m.senderStyle.Render("You: ")+userMessage)
// Update the viewport content immediately
m.viewport.SetContent(lipgloss.NewStyle().Width(m.viewport.Width).Render(strings.Join(m.messages, "\n")))
m.textarea.Reset()
m.viewport.GotoBottom()
// Return the command to process the output
// Will always return a responseMsg (or Titan response in other words)
return m, processOutputCmd(userMessage)
}
case responseMsg:
// Handle the response from the Titan model
if msg.err != nil {
log.Fatal(msg.err)
}
m.messages = append(m.messages, titanStyle.Render("Titan: ")+msg.response)
m.viewport.SetContent(lipgloss.NewStyle().Width(m.viewport.Width).Render(strings.Join(m.messages, "\n")))
m.viewport.GotoBottom()
...
1. First block we're defining a response struct to hold our response from ProcessOutput() coming from the titanModel.go file.
2. Second block is the function to run every time a user presses the Enter key, after they've input a prompt. You'll see that in the third block. This block specifically calls the ProcessOutput() function from titanModel.go file. Remember this is where the actual BedrockRuntime communication happens.
3. Third block is from within the Update() Bubbletea function. Notice when the KeyMsg is a KeyEnter (user pressed Enter), it takes the users input and appends it to m.messages then calls the processOutputCmd() function. This function is called every time Enter is pressed so we're assuming that the user input was valid or that they even entered something. This code for sure lacks validation but its just a proof of concept.
Run Code
After you've cloned the repository and been granted access to the Titan FMs in AWS, do the following to run the code.
- Run a
go get .command from the root of this project to install all the external packages
go get .- Get into your AWS console and create an Access Key. This Access Key should come from the account that has access to Bedrock and been granted use of the FM's. Documentation for creating some from the root user can be found here. Quiet Storm and AWS do not recommend using Access Keys from the root user. Take note of the key ID and secret access key.
- Export your Access Key credentials with the following environment variables, example:
export AWS_ACCESS_KEY_ID=C39ENCQ3EGINC93UBEWOUI
export AWS_SECRET_ACCESS_KEY=EROVNROhI6FC39RCJ9RJX0aXzYCSDCSDVCcV0lUYWve0Sp- Run `go run main.go` to run the main() function directly or compile the code into an executable, an example:
go build -o chatbot- Run the executable and prompt the Titan FM from Bedrock!
You should have seen something like this:
