Generative AI – Doodle Coach App

In this tutorial, we will create an app that uses the Google Gemini API to describe a hand-drawn picture and suggest improvements.

You can download the source code (aia file) from the download section at the bottom of the page.

For calling the Google Gemini API, we’ll use Google Apps Script (GAS). The GAS script is written with Google Gemini 2.5 Pro.

The Gemini API created in Google AI Studio can be used for free with Gemini 2.5 Pro within the following limits:

• Requests per minute (RPM): 5
• Tokens per minute (input) (TPM): 250,000
• Requests per day (RPD): 100

What You Will Learn in This Tutorial

• How to create an API with GAS
• How to write a GAS script using Gemini
• How to integrate with the Gemini API

Components Used

Button, Label, TextBox, Canvas, Image, Web, ImageBase641

Blocks Used

Global Variable, List, if then else, Dictionaries

Setting up Google Apps Script

Set up the server-side program in Google Apps Script to make API calls.

Generate a Google Apps Script Using Gemini 2.5 Pro

You can use Gemini 2.5 Pro from here even with a free account.

Prompt

I want to add descriptions to the drawings made on the App Inventor canvas and have it suggest improvements. I’ll use Google Apps Script and Google AI Studio. The drawing on the canvas will be base64-encoded with an extension and uploaded in JSON format. The result will be returned in JSON. The API key will be read from the script properties. I’ll handle the App Inventor part myself, so please explain the rest.

Response

Of course. You can set up a Google Apps Script web app to act as an intermediary. It will receive the image data from App Inventor, forward it to the Google AI API for analysis, and then return the structured results.
Here’s a step-by-step guide to setting up the Google Apps Script and handling the AI interaction.

Overview of the Process
1. Get Google AI API Key: You'll get a free API key from Google AI Studio.
2. Create a Google Apps Script: This script will contain the code to handle incoming requests.
3. Store API Key Securely: You'll save your API key in the script's properties so it's not exposed in the code.
4. Write the doPost Function: This function will receive the image from App Inventor, call the Gemini API, parse the result, and send it back.
5. Deploy as a Web App: You'll publish the script so App Inventor can send it requests.

## Step 1: Get Your Google AI API Key 🔑
First, you need an API key to use Google's generative models.
1. Go to Google AI Studio.
2. Sign in with your Google account.
3. Click on "Get API key" and then "Create API key in new project".
4. Copy the generated key and save it somewhere safe for the next step.

## Step 2: Set Up the Google Apps Script Project
This script will be your backend service.
1. Go to script.google.com and create a New project.
2. Give your project a name, like "App Inventor AI Helper".
3. Store your API key securely:
  Click on the Project Settings ⚙️ icon on the left sidebar.
  Scroll down to the Script Properties section and click "Add script property".
  For the Property name, enter GEMINI_API_KEY.
  For the Value, paste the API key you got from Google AI Studio.
  Click "Save script properties". This keeps your key out of the code itself.

## Step 3: Write the Apps Script Code ✍️
Now, replace the default code in the Code.gs file with the following script. This code handles incoming POST requests, which is how App Inventor's Web component sends data.

function doPost(e) {
  try {
    // Parse the JSON data sent from App Inventor
    const postData = JSON.parse(e.postData.contents);
    const base64Image = postData.image; // "image" key expected from App Inventor

    // Load API key from script properties
    const API_KEY = PropertiesService.getScriptProperties().getProperty('API_KEY');
    if (!API_KEY) {
      return createErrorResponse('API key is not set.');
    }

    // Gemini API endpoint
    const API_URL = `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent?key=${API_KEY}`;
  
    // Request body to send to Gemini API
    const payload = {
      "contents": [{
        "parts": [{
          "text": "Please describe this drawing and suggest improvements. Separate description and suggestions clearly. Limit to 100 characters."
        }, {
          "inline_data": {
            "mime_type": "image/png", // App Inventor Canvas produces PNG format
            "data": base64Image
          }
        }]
      }]
    };

    // Request settings for Gemini API
    const options = {
      'method': 'post',
      'contentType': 'application/json',
      'payload': JSON.stringify(payload),
      'muteHttpExceptions': true // Capture error responses instead of throwing exceptions
    };

    // Send request to Gemini API
    const response = UrlFetchApp.fetch(API_URL, options);
    const responseCode = response.getResponseCode();
    const responseBody = response.getContentText();

    if (responseCode !== 200) {
       return createErrorResponse(`API request error: ${responseCode} ${responseBody}`);
    }
    
    // Parse response from Gemini API
    const resultJson = JSON.parse(responseBody);
    
    // Extract generated text from response structure
    let generatedText = "Failed to retrieve analysis result.";
    if (resultJson.candidates && resultJson.candidates.length > 0 &&
        resultJson.candidates[0].content && resultJson.candidates[0].content.parts &&
        resultJson.candidates[0].content.parts.length > 0) {
      generatedText = resultJson.candidates[0].content.parts[0].text;
    } else {
       // Unexpected response format
       return createErrorResponse(`Unexpected API response format: ${responseBody}`);
    }

    // Build JSON response for App Inventor
    const output = JSON.stringify({
      "status": "success",
      "result": generatedText
    });

    // Return result in JSON format
    return ContentService.createTextOutput(output).setMimeType(ContentService.MimeType.JSON);

  } catch (error) {
    // Catch unexpected script errors
    return createErrorResponse(`Script error: ${error.toString()}`);
  }
}

/**
 * Helper function to create an error response
 * @param {string} message - Error message
 * @return {ContentService.TextOutput} - JSON object containing error information
 */
function createErrorResponse(message) {
  const errorOutput = JSON.stringify({
    "status": "error",
    "message": message
  });
  return ContentService.createTextOutput(errorOutput).setMimeType(ContentService.MimeType.JSON);
}

## Step 4: Deploy the Script as a Web App 🚀
For App Inventor to be able to call your script, you must deploy it.
1. Click the Deploy button at the top right.
2. Select New deployment.
3. Click the gear icon next to "Select type" and choose Web app.
4. In the configuration settings:
  Description: (Optional) "AI image analyzer for App Inventor".
  Execute as: Me (Your Google Account).
  Who has access: Anyone. This is critical. It makes your API public, but it's necessary for App Inventor to access it without a login.
5. Click Deploy.
6. You'll be asked to Authorize access. Follow the prompts, choosing your Google account, and click "Allow" on the warning screen (it's safe, as you wrote the code).
7. After authorizing, a Deployment ID and Web app URL will appear. Copy the Web app URL. This is the URL you will use in your App Inventor project's Web component.

App Inventor App

From the [Projects] menu, select [Start new project] and name it “DoodleCoach“.

Designer