0:04 [Music] hi there everyone my name is Paige Bailey and I am the lead for our AI developer experience team at Google um today I'm going to talk to you a little bit about AI studio and our Gemini apis and how you can get started using them with all of your projects um I think you probably heard a little bit about them earlier today um but this session will be specifically focused on kind of a deep dive into AI studio um and using all of the great features that we have within the UI so with that let's get started um generative AI is really transforming how Google is building all of our products uh and you've probably already seen this with something like um
0:50 you know Gemini being baked into Google Docs into Google Sheets um being uh baked into code completions code generation and finding and fixing errors within collab um and pretty much every single one of our product surfaces um and you know Google isn't really a stranger to machine learning and AI our company's entire Mission around organizing the world's information and making it useful really really ties into machine learning and AI in general and we've been doing this a lot um since uh you know uh early in the earlier in the lifetime of Google um and then also uh even for things like tensorflow's first open source release in 2015 um as well as great projects like Al Alpha zero and Alpha fold from our colleagues at De mind um and then finally Gemini which is
1:37 our latest Flagship model that we'll be looking at today uh Gemini is multimodal from day Zero which means that it can understand videos images audio text full code bases uh and it can do really really interesting things with all of the all of the great knowledge that it's baked into its uh that it's been baked into its training process by the Deep mine team we have several versions of Gemini that you can try out today for free in AI Studio including Gemini 1.5 flash which is kind of a production ready version um very quick very Speedy um very costeffective as well as Gemini 1.5 Pro which is our most capable model that's been released externally it has up to a two million token context window and we'll be getting into uh we'll be getting into what that means in just a
2:26 second um uh while also being uh you know top tier across a broad variety of complex tasks we've also just recently released flash 8B um which is an even tinier version of uh Gemini 1.5 flash um that's super quick that's really really capable especially when coupled with things like code execution and that can do a lot for your work um these models have been performing pretty well so when Pro was first released it was at the top of the LM CIS leaderboard um and you can also check out uh most present day stats on artificial analysis. um you'll want to be in those green quadrants and so you'll see here that flash when it was first released created its own quadrant um and you can see how Gemini 1.5 pro has been
3:11 improving over time I think now it's it's very very close to this top left uh leftmost quadrant um which indicates kind of the the best quality versus price tradeoff uh that that you can have for the models themselves um as I mentioned we're deploying them everywhere throughout all of Google's product surfaces but the very very small so things like phones and mobile devices as well as baking the models directly into the Chrome browser um and we're integrating them also into things like docs and sheets and slides and meet um we talk a lot about context Windows why they're important why they're exciting um but we don't do a really great job of explaining what uh what a 2 million token context window allows you to do um today I'm going to show you a couple of examples of this but what 2 million
3:57 tokens in uh your context window might allow is you can suddenly get the model a lot of information WR it in friends time so instead of having to do things like fine tuning instead of having to do things like standing up Vector databases for retrieval you can just kind of give the model a lot of information um and it's Off to the Races all at once um so it's very very cool to be able to see what these models can do with longer context Windows um and you can stuff an awful lot of information in them as well so uh 1 million tokens is really all of the emails that you've sent in the last year um every text message that you'll send in a lifetime hours and hours of video um many of the many of the papers that are published at nups this year um
4:43 as well as uh you know hundreds if not thousands of songs um so just imagine what you can do just being able to give Gemini all of that information all at once um and to be able to see how it uses it uh uses it in real time so with that I'm going to go ahead and get started with some of our demos I'm going to take you directly into AI studio and hopefully all of you have seen AI Studio before um if not it's very simple to get to um at aist studio. goole.com um and all you should have to do is log in with your Gmail account so the the Google account that you use to log into all of our services um is the one that you'll be able to access uh uh in AI Studio you'll see here a couple of example
5:28 prompts that you can kind of click and use um we also have a prompt Gallery where you can explore different prompt ideas um to help you get started um but for today I'm going to go ahead and and take a look at one of our uh one of our outof the-box data uh uh one of our outof thebox data assets um and we'll use it to investigate some of our models so if I click here this plus sign you'll see that I can access data that stored within Google Drive um I can upload files I can record audio or even record video videos directly within the UI itself um uh take pictures or use some of our sample media and for the purpose of this demonstration I'm just going to use some sample media that we've already
6:13 uploaded I really love dinosaurs so I'm going to click this video right here um click add it to the prompt um and then I am going to use Gemini to help analyze it um if I click here under model categories you can see Gemini 1.5 Pro which is that most capable model that I was mentioning before um as well as flash and flash 8B um and then also our Gemma models which are a family of open- source models um for this video analysis I'm going to choose the latest version of flash um you'll see here off to the side that it's almost 89,000 tokens for this 5 minute long video um and what I'm going to do is I'm going to say please um tell me uh the names of all of the
7:00 dinosaurs uh in this video um uh and and uh also give me the timestamp for when each dinosaur skeleton is shown um also make sure uh to give me a fun fact uh about each dinosaur um and then I'm not very good at prompt engineering so so this uh hopefully this ends up working um if I hit run um immediately Gemini starts plugging and chugging with the with the latest flash version to analyze the video um and to uh to kind of look through all five minutes of this video footage uh to help get the information that it needs in order to answer my question um you can see here that it
7:47 only took 14 seconds to analyze the full video um and you can see that uh it's spotted in ornithomimus and a couple of segments of the video uh dyus uh struthiomimus and Allosaurus and Apatosaurus um uh and has given me fun facts about each one um I especially love struthiomimus the the ostrich mimic um just because it it helps us kind of understand uh that dinosaurs are are just kind of the naturally evolved state of birds um so this is very very cool and as I mentioned super super Speedy um for uh for the smaller version of Gemini 1.5 fl8 um one of our uh one of our very good my friends externally Simon
8:33 Willison recently did an experiment where he analyzed 35 seconds of video for just looking over his bookshelf um and calculated that it took one 100th of a penny um in order to do that analysis So in theory you could record everything that's happening on your uh on your screen all day every day 365 days a year um and it would still cost less to analyze than probably what you would spend in a week on Starbucks um so it's it's very very cool to see uh it's very cool to see how all of these um capabilities are baked into the the UI for AI studio um and also how you can kind of analyze videos uh analyze videos
9:18 in real time um in addition to video uh you can also uh you can also analyze um uh you can also analyze PDF documents so you can see here I've uploaded the first book in the anamorphs series hopefully people uh have heard of anamorphs if not then uh then I strongly strongly recommend you taking a look it is one of the uh the coolest series I remember from my childhood um but you can ask for things like please transcribe all of the text um from page 66 of this PDF and it gives you a transcript outright if you want to be able to use um some of the great features that we have here off to the right like code execution Json mode for structured outputs function calling and grounding
10:03 um you can also uh sort of experiment with those settings as well as our safety settings um so in general I try to turn most of my safety settings off as I'm experimenting which gives me the ability to to kind of um prevent some of the problematic outputs that you might see from the model um for this next example I'm going to show you how you can do uh uh so as an example if I click Gemini 1.58 um I'm going to show you how you can do interesting things with search grounding and also with code execution um in order to get more appropriate model responses um so as an example if I say something to the effect of my birthday um is September 21st um everyone should please send presents um
10:49 but uh please tell me uh which day of the week my birthday will be on uh from uh 2025 to 2525 um if I just ask the model outright probably not going to give me a great response so it's it's saying I can't give you a complete list of the days of the week without access to a powerful calendar program um however what you can do by turning on code execution is give the model access to precisely that to an isolated sandboxed environment um with python the standard library for python and a whole bunch of additional libraries that will allow you to to kind of run this again um I I I uh that will allow you to to kind of run this again and let's uh let me pull up the Gemini
11:37 flash uh model just to to see what can Happ there we go um and what it does is it kind of figures out the day of the week um it creates some python code in order to calculate this um and then uh runs the python code for me um so you can see here this is the python code this is the expressed output um and it tells me which day of the week my birthday will be on um from now till uh till the very end and so again if I turn off code execution um and then try running with the with the flash model um the model says no um if I turn on code execution um the model writes some code and then tells me which day of the week my birthday will be on just like that um
12:24 which is pretty magical um relatedly you can also ask questions um uh so I'm going to to remove this and I'm going to change my prompt um I'm going to say something to the effect of please tell me uh who won the physics Nobel Prize in 2024 uh and so if I click stop editing um and then try to run this um with with no additional kind of bells and whistles um what uh what should end up happening is that the model tells me that it can't know because this has passed its training data cut off or um the uh or it will just fabricate or hallucinate a response so if I hit
13:09 run um yes it it says uh unfortunately I don't have access to real-time information if you turn on grounding um so grounding with Google search um and then you run again and what you can see is that the model is capable of synthesizing uh the top Google search suggestions um and then also grounding it with sources um including uh including search suggestions um so this is pretty magical um even more magical is if you click this get code button off to the right you can see the code that you would use in order to replicate each of these experiences kind of automatically created um uh based on whatever you had just accomplished in the UI as well as
13:55 giving you the ability to open up this example in collab um so it takes you immediately to uh to a nice notebook um with uh kind of the the content that you would need to generate as well as the the conversation um the conversation that you would uh that you had been working with um you can also see here off to the left if you wanted to add a secret um you could have a Gemini API key created um or you can uh sort of import your keys from AI Studio here directly within cab um so it's it's pretty beautiful how all of this is is kind of encapsulated together um uh H and uh We've even added just recently
14:40 the ability to uh if you prefer using the open AI API libraries um we've given you the ability to import the open AI Library um and to use the Gemini models directly from there um so instead of uh instead of having to to kind of use the open AI Keys you can just use the you can just use the Gemini versions instead um for function calling um I strongly strongly recommend you play with function calling not just for tools and not just for information extraction um but also by even being able to uh to use different models or different kind of like satellite imagery segmenters um you can uh sort of pull in uh pull in
15:25 your question um ask for Gemini to dynamically understand which tool it should select in order to help accomplish that task uh and give the response back to you um so lots of really interesting um work that you can accomplish with uh with function calling if you'd like to generate a key yourself um we already saw how you could generate an API key directly within collab um you can also click get API key here off to the left um and create a new API Key by clicking on this button um you can also create tuned models if you would like to to use Gemini itself uh uh with tuned data um so here you can upload an arbitrary CSV or an automat uh an arbitrary data
16:13 source um change some of the settings um including the number of epics that you might have for tuning um and tune your model uh and then call it like an API just as you would any other Gemini API so lots of really interesting things that you can do directly within AI Studio you can also export the code for whatever you've just created um and use it uh use it to enrich all of your applications with Gemini whether you're using python or JavaScript or or anything else um I also would like to call out that Google has an AI for startups program um that allows you to use uh Google Cloud credits to build the systems uh build the systems that your that your company is is using um or the applications that your company is
17:01 creating creating um we have a kind of a a generalized starting tier so if you're prefunding you might qualify for for start or for ecosystem and then once you uh kind of scale out um uh and and get a little bit of funding you can get um Equity free uh pretty significant Cloud resources um to date so with that I'm going to say thank you so much if you remember one thing from this presentation it is to go to try out Gemini 1.5 um in aist studio. goole.com again completely for free both for the API as well as the UI interface and if you have any questions whatsoever as you're building out these projects um please feel free to contact me at webpage google.com we're so excited to see what
17:47 you build um uh and really really jazzed to have you um to have you dialing in today to all of the great presenters thank you so much uh and I'm looking forward to uh looking forward to the rest of the conference [Music]
