Learn AI Programming with Python| Python for Artificial Intelligence | Edureka | DL Rewind-5

0:10 hi everyone good morning good afternoon good evening based on the time zones and guys before starting the session can any one of you give me a quick confirmation you can see my screen and hear me as well great thank you for the confirmation so my name is sham i have been working with it industry for close to 10 years now so before i start let me uh share a edu masterclass community with you all so right now what i'm showing was in projecting is educate up wherein you know this is a kind of

0:55 community of master class we started back in 2019 and since then we have been closing into almost you know 30 000 members so far and about in this month in these master classes we published you know more than 100 webinars and live events on different topics like iot big data ipa aiml pmp and multiple backend and front-end based technologies like java php all those technologies we are covering in the webinar so that you know we all should be aware of what all technologies are going on and you know the best part of these trainings from edureka is these webinars are absolutely free

1:41 this platform will help you to give a direction the way you want it to get started and achieve you know your goals and to reach the destination organizations basically you know you will be notified uh the best part is you know i mean why to join this group is uh you will be notified about the schedule recordings i guess most of the members already have been enrolled and they're benefiting about uh these uh you know from meetup platform so we'll start what is artificial intelligence you know that's a word which is from you know past you know from five years that that's been you know um into uh more into the iit sector you know it is going into pictures so artificial intelligence you know it was uh introduced late back in 1956

2:27 1950s but what exactly it is you know artificial intelligence in in the technology in the computers basically it's a wide-ranging branch of computer science concerned with building smart machines capable of performing tasks that require typically human intelligence so you know in a nutshell i can say simulating human intelligence and machines that is all about artificial intelligence so example i can give you some of them you know speech recognization which we can see in our environment right now everything you know you you say we have cv we have uh okay google all those platforms which are uh indirectly or directly using ai platforms only

3:13 problem solving skills you can make your model learn about these kind of aspects and so that so that it you know if you train someone you know to do a repetitive job or a sensible job as well so ai model is capable of doing catering this particular solution and learning and planning as well you know learning and planning when i say we have a very good example of planning you know there are some project plans which you build up uh you feed some of the from you know past how many i guess so many projects we have done till now you feed all those models into this particular ai platform you will be able to uh from the inputs which you provide you will be able to predict that this particular project how much time it will

3:58 take including if all the you know the session the phases of the project model either it's a software development life cycle or else it's a normal projects which we use in any of the industry okay so we're giving you some of the examples you know it's like a silly uh iphone i mean everyone is using um if someone is having you know apple iphones and you know how siri and okay google like using this item intelligence whenever you ask some questions you schedule some of the meetings you um set an alarm and your you know your sleeping time your uh you know a basic basic methods which um all those things that data is getting captured in your phone itself based out of which prediction technologies goes on

4:45 then we have a tesla like autopilot mode and cars uh cogito is a company you know provides a real-time guidance to the customer believe me this platform is like uh they're giving 70 of the customers based on ai they're giving answers over the call and uh they're giving a quick answer to the customer and nobody can understand the human speaking or artificial intelligence models are speaking those answers netflix recommendations based on consumers reaction interest choices and behaviors you know this is also part of ai then we have multiple examples pandora nest flying drones eco eco dot you have heard about amazon product to schedule

5:30 appointment control lights answers questions all right so um let me quickly jump out to this i mean what all agenda we will be covering today that is introduction to artificial intelligence why python for ai features of python python packages and we'll be covering some of the hands-on examples as well where again we'll show you know how the data modeling can be done based out of which you can predict do some predictive analysis as well for the data which is already present in the market so you know before we start with the particular agenda i would you know i would just wanted to uh segregate a differences or else you know uh between most of the people are you know right now confused what is ai what is ml what is deep learning so setting this ground ground rule that is

6:17 ai is a science of getting machines to mimic the behavior of human that's a ai mimicking you can say a straightforward mimic is directly proportional to the ai then we have ml ml is basically you know machine learning subset of ai that focuses on getting machines to mode the seasons and feeding the data you know you feed the data and there is data modeling goes on i mean the you know very branded example i can say now that's a apple example you feed 1000 uh colors of apples to this machine learning models then whatever you feed uh to the machine learning models that will be able to predict it's an apple or it's in guava you know any other fruit so based

7:03 on the data which you have given to the machine learning so there is no i mean i'll tell you like machine learning is not having something you know which uh automatically it is enhancing the skills it is taking up the decision whatever you have given to the machine learning models that only it is able to capture and give you a predictive results and there is another term deep learning deep learning is basically a subset of machine learning only that uses concept on neural network and solving the complex problems based on some mathematical rules which you know which are having um having python in in our technologies which we have right now numpy you know which just a base calculations based out of the climatic or stock market whatever the data has been fit

7:49 okay introduction to our artificial intelligence i guess i have covered most of the things from what is artificial intelligence so i mean go for this particular model you know what is why there's a demand of you know ai so right now if you see there's you know maybe there's uh tega bytes of data you know gigabytes of data which is flowing in a second a fraction of seconds so you can you can see like how much data has to be evaluated how much data has to be processed to get a meaningful information in seconds of timelines that's what you know there is a demand of ai and you know we are in ai we are talking about not about talking about cpu we are talking about gpu here so you know in if you have to you know model something you have to create some

8:35 models you have to feed the data you have to thousand you know color of data you have to feed to some of the models and then you have to be able to get the data outside that so in this particular scenario you know you will be able to compute data in fractional seconds by using gpus then better involve algorithms by using python you know python is having right now we'll be covering with more you know more libraries like numpy we have matplotlib multiple libraries which through which we have a better algorithms no need to write the code thousands of lines of code already been built up with the for you guys and you have to just you know plug and play and you know broad investment more data big data you know big data is something you know you are calling to someone even that is a mean you know

9:21 whom you are calling how much for the frequency based out of which you'll be getting the packages as well as offer from telecom sectors these are the big data analysis has been done okay what does ai have covered with that so we have a pillars for ai i'll go one by one you know for all the pillars what exactly the machine learning deep learning i've given you you know the the basics of machine learning but i'll still you know explain you why what does machine learning okay all right so uh basically in you know in artificial intelligence if we talk about any algorithms it's a sequence of you know statistical uh data processing steps so if you want to make some decisions you have to uh write an algorithm and

10:08 you know you have to train to find the patterns and features in massive amount of data likewise i give an example of apple's you know if you feed green apple red apples all those things i mean if you feed like thousands of apples colors to one particular model the machine learning models are so super fast resilient that it is able to predict the future i mean what all things can be predicted and you know it's a basically a branch of artificial intelligence itself and focused on building application that learn from data to improve their accuracy over that over the time without being programmed or do to do so so it's like you know means something if you if i give you a real-time example so i'm talking about jpmorgan

10:53 so jp morgan used to spend 36 000 hours of employee time just to you know figure out what is written in the particular um sheet or you know pdf files are kind of invoices they were having they was having some pos and per pos and po orders they were having and uh you know based out of that particular uh machine learning model they have trained they have given training to close to you know samples close to uh i guess from 1000 3000 models they have given how how the invoices will be looking like now they're able to process uh close to 70 000 records of pdf data that the scanned images

11:39 basically you know scan the images having you know it there is ocr technologies which is revived integrated and also along with that there is analysis on the basis of what where are the details getting who has written those details and though on those basis they are able to extract the data and insert into some of the erp system to get the data in you know particular manner and you know the main thing is is not only saving 36 000 hours that is also you know if you write you take data from somewhere and if you write in the system it's it's a uh it's a tedious job to do and you know and even there is a spelling mistakes errors all those things gonna happen but uh now they have getting a qaco 99 person that's a commendable you know

12:25 machine model example which right now you know these companies are using then we have nlp nlp is you know uh i'll give you an example like uh if you write some emails right now in go in gmail there is a spell checker there's auto complete spam filters uh voice text messaging also there's the features of you know these are all natural language processing so how i mean this is a branch of artificial intelligence itself you know many of us are using nlp on a daily without realizing like you are probably used at least once you know and all those tools right now as i told you example and i'll tell you like you know in artificial intelligence how it is used as when you're writing some email you know so it's not only how you are

13:12 writing so tons of data mails which are getting exchanged on a daily basis from those only it is uh gmail is picking and doing some you know uh processing has been done for the national language and and how the people of which region are writing you know uk has different aspects of writing us as different aspects of writing is able to process and computer is able to a ability to read and understand to interpret human language as well now through this particular nlp as a tool then going to deep learning so i'll explain you like you know i mean how deep learning is also evolving in some certain criteria you know deep learning is uh an ai you know i told you like there

14:00 is machine learning then also deep learning is going involved so deep learning is a subset of machine learning only in which multi-layered neural network you know model to work with human brain and learn the large amount of data through this particular deep learning aspects and you know deep learning uh i can say in a nutshell it's a kind of simulation of human brain how you think so based on you know the modeling uh let's take an example today you are telling the model that is you know a plus b is equal to c so you can have some permutation combination that that is called as machine learning now you know automatically there will be a predictive analysis by ai and deep learning together to get you know multiplications as well division as well and there can be another introduction as

14:46 well from that is from the ai aspects so that is how you know um without human intervention as well uh automatically through deep learning the the accuracy of the model which you have trained in machine learning uh if you if you engulf deep learning it will be able to increase the accuracy based on you know objects and complexity tasks whichever you have provided to that particular model okay all right so um going to the other topic that is computer vision i'll give you only a two minus or three that's you know um computer vision needs uh even it's all about data only you know machine learning nlp deep learning computer vision all about the data itself so but

15:32 it's it it computer vision only runs it runs on you know analysis of data over and over until it uh decides the distinctions of ultimately recognizing images basically i'll give you a very good example here you know if you watch some of the movies if you watch some of the videos in youtube maybe you know the name is different so but also if that computer vision is enabled there automatically the images are getting captured like you maybe you like some bollywood star you like some some hollywood star you will be able the computer will be able to extract the information based on the pixels and it will be able to give you a recommended solution on so basically recommended solution or

16:19 you know recommended results whom do you like and based out of which you will be able to you know um get some more videos that's the basic example you know and so basically if i talk about that apple example the favorite example of you know all the machine learning and ai iii person that is you know uh to train a computer to recognize when apples you know it needs to be fit vast quantities of apple images and apple related to items and learn differences on recognize apple so that you know um if someone is getting some other region apple so it will be able to identify what particular reason is also you know it is stating so that you know there's one to one or one too many mapping as well okay all right so that's so you know i mean more basic four pillars are there the knowledge base is something you know which we have already

17:07 uh in our system in which you know if you are training someone you are creating some libraries as well and also you are creating a good knowledge base so it's like you know um you have some inference system and uh so it's like it's a kind of intelligent you know there's environment and then the influence engine is there so that you know there is a knowledge base based out of which you are able to make the machine learns that's how the knowledge base works you know that's a repository kind of you know uh method you can say so that the maintaining on internal state of knowledge reason over that knowledge update their knowledge and observations and how they have to take their actions based on this knowledge base

17:53 okay all right so i think these pillars are fine uh there is last pillar that is you know expert system in which basically it comprises of the system in which um we again it's artificial intelligence itself and an expert system incorporates a knowledge base containing accumulated experience of an influence rule engines how machine learning can also be trained and how you know if something is repetitive how it can be trained to this particular model so that you know it can simulate the judgment and behavior of a human or an organization that has expert knowledge and experience on the particular field or particular domain okay that's uh introduce introduction of

18:39 artificial intelligence when it was done it took from john mccarthy and uh he he only introduced artificial intelligence editor that was a time in which you know nobody have predictive so but now we can see there is most of the systems which we are using as artillery intelligence okay so now we have types of ai so you know there are some types of air there are three types of ai basically artificial nago intelligence then we have general intelligence and the other one is super intelligence so basically you know i mean right now i'll say narrow is something which we can say it's a basic level of intelligence it is also called uh as you know a week ai that involves

19:26 applying ai only to specific tasks you know limited predefined functions even no genuine intelligence or self-awareness despite of being sophisticated example i can give you some of the examples which you know which you can late in the daily life that is face verification in iphone that's a week ai you know on if you have seen that particular model which basically steve jobs and the owners of apple have guided with machine learning technology which they are using they are using you know very small uh sector of the ai in which you know face is verified based out of your um your pixels of your face and you know that is based on a dot mechanism

20:12 and there's autopilot feature uh social hum humanoid if you have seen you've heard about sofia so i mean that's a sofia bot that is a week ai is basically example google maps google gives them the suggestions there are some you know sometimes google maps also details you know how which which basically route you have to take to reach to some destination so that's basically you know narrow intelligence which currently we are in in current environment but second thing is you know general intelligence that we can say the strong ai that involves machines processes with you know ability to perform any intellectual task that human can you know right now it's it's a kind of solving riddles

20:58 and the other example is playing chess you know there are some other models which already been developed even if you play some of the games you know on the daily basis whatever games you are playing you know whatever inputs you are right now doing based out of which the data is going and feeding into their database and they are even you know data analysts who are sitting on the other side they're even watching the data like which particular stage uh the user is getting failed you know for four times five ten days out of which they're even changing the games algorithms so that you know uh you do not lose the interest and you play the game again because they are getting profit from by you are downloading their app and you're seeing advertisement on in their app so i mean i'll tell you one quote from stephen hawking also said you know so

21:44 right now strong ai would uh take off on its own you know a human will be superseded just because human involvement is only done on the basis of biological updates you know if you wanted to update from yourself that's a human is going face by phase and it'll be ten years there is some other brains you know if you see right now basic example like an example that's the kids right now they're using phones as well so that's how it is changing human involvement but sooner you know ai will be super evil will be you know superseding all those aspects so that's in artificial intel general intelligence which is right now we are uh introducing still on a very you know a basic stage which you say and the third one is you know artificial

22:30 super intelligence if you talk about that's more or less um time when the capability of the computer will surpass the humans so it's a kind of hypothetical situation i can say and you know it's only depicted on some of the movies if you see some of the movies in some hollywood movies or some bollywood movies as well there is you know there are some artificial intelligence which can supersede uh human brains as well so that is artificial super intelligence okay so our main target that is a human why python for ai so i mean there are a few other things which uh python why only python we are using so first of all less code you know from the previous

23:16 legacy technologies uh for for even for doing one task you have to write 10 lines of the code but right now it's only extremely less code you have to only focus on your algorithms the first point and you know python has pre-built libraries like numpy for scientific computation and for scipy for advanced computing and pi brain for machine learning these are the already given pre-built libraries which has been developed by some of the um you know some of the googlers or some other hits and already they have introduced these libraries you have to just simply use their libraries and get set go you're ready for your pre-built models i'll show you the example live example as well

24:02 and python is also a platform which independent and hence one of the most flexible and popular choices for use different platform technologies i mean so basically if you are coming from different background of the coding so it's it's you know you are a very guy down very right track because you know now you need not to worry about the coding basic syntax because python is extremely easy as compared to the previous models and also you know uh python is platform independent i'm using mac you can use windows all those independent platform independent um so python is also flexible for all others with options of choose between you know we wanted to choose object oriented programming language approach or scripting based you know because why

24:49 i'm saying this is you know there are two types of terminologies here object oriented programming language another is scripting base so some guys are who are coming from the background of javascript and you know jquery's angularjs i would suggest i mean they can greatly go to you know the scripting based methodology in python as well it will be easy to transition in this particular manner but somebody is coming from the java background or you know totally indoctrinated background then choose a oops approach that is object-oriented programming approach then python is having a massive community right now you i mean there's a huge community as you see of community as well here also it's a work like the same but more or less you can use you know python developers around world provide you comprehensive support

25:35 and assistance i mean believe me i mean you write some of the queries you will be getting answer in another 10 or 15 minutes maximum anyone will be able to apply you and solutions that's how the massive community is working right now for the python okay all right so uh python packages i mean uh today i will be covering some of the packages what all has been used what are we can use for the python packages here so that a package is something you know i mean it's like package is already a developed code which you have to use other plug-and-play components so it's a kind of reusable components you have to pass some of the parameters you have to remember some of the functions you have to pass the parameters based out of which all algorithms are getting

26:21 into this particular packages and you will be able to easily use okay so uh the first package i will be uh package and library are the same thing you know do not confuse on these two terminologies one is you know what is tensorflow tensorflow library were developed by google and collaboration with brain team and popularly for machine learning algorithm again a subset of aisl and currently you know tensorflow is the most famous deep learning library in the world in google and tensorflow so what happens is you know to give you a concrete example google users can experience a faster and more refined the search with ai if the user types keyword a search bar

27:08 google provides a recommendation about what could be the next word that's how you know i'm and you know that's a tensorflow beauty only so currently it is using it as a tensorflow mechanism itself you write something you know you write something you know what is this you will be able to i mean sometimes you have observed like yeah this is what we we want and they are giving us the results so i'll go you one by one you know all those things so google wants to use machine learning to take advantage of this you know massive data sets so uh we have three different groups in machine learning one is researchers then we have data scientists and we have programmers so researchers who research you know what all uh data we are getting and based out of which they are feeding to data scientists and the programmers are basically

27:53 giving a proper coding mechanism to that particular search and you are able to then get the examples on this particular search results basically on and yes basically on the basis of search engine auto optimization as well that's another technology okay so how you know the tensorflow architecture works i'll give you on units basically a three uh layered model you know in which there is one is pre-processing of data what is pre-processing pre-processing is basically you get some heterogeneous data you get some data of you know let's say an example cars you get toyota you get bmw you get audi you get uh hyundai all those data you get and you pre-process these data you know

28:38 what exactly you need you know you have to first write your problem statement so your problem statement can be uh maybe uh your organization is predicting like toyota will be doing a great job in 2025 based out of which parameters so you have to reprocess those parameters maybe they have more multiple units maybe they are collaborating with mahuti maybe they're collaborating with another another one and they have a very good projects you all have that data and then you know you can go and confidently tell to your management see boss this this cannot be happened based out of this particular data so either you have to change the plan either you have to you know change the strategy but we cannot predict on

29:24 these basis and this predictive analysis is so important for the you know stock exchange as well you know they're uh based out of uh those mechanism only stock exchange also works and you know you have to invest or you do not have to invest you so these tensorflow you know right now it's been worked here the other uh i mean i told you the first one is pre-processing of data the second one is building the model you know the model which you create uh you align the data you separate the data of toyota you separate the data on other companies you see the project protection of the data all those things in a building or the particular model and then you have to train and estimate the model training uh in every machine learning and ai model training is an extremely

30:10 important part because uh each and every model which you develop with what you have to get the output one by one you have to go use this particular library and see from your training results for the particular let's say example we are sixty thousand or one lakh records you cannot do you know and this is on all sixty thousand records or one that records so you choose some other data like say one 1000 records you do some unless say some random 1000 records then you impose all those uh random records to next 60 000 records to get a particular particular resultant that's you know it is called as tensorflow because uh it takes input from multi-dimensional array and array is basically you know a kind

30:56 of numpy you can say and numpy in python in which you know there is an array in which there is some computational going on and there are those bases you have to pass some of the numeric data and that is called as tensors in this specific tensor architecture it follows all right so i think tensorflow is majorly into you know why this has been used or tensorflow is uh because the tensor goes into the flows through a list of operations and then it comes on the other side to predict the data what can be happened and what strategies has to be taken in another place okay and then i mean the most important it can be it's again the platform independent it can be in any of the os but the only thing is your you should

31:41 have a memory of at least 4 gb ram and you know a basic qualifications to avoganda software all right and going to the next top okay so these are the features responsive flexible easy trainable parallel neural network training okay so now next uh next slide really again and this is uh scikit it's a scientific kit you know in which again it's a library which you have to use as i import package in python and basically unsupervised learning algorithms and cross validation i'll give you a very good example here you know uh supervised and unsupervised i'll give you a one example supervisor's learning is basically

32:26 let's say a email which you're receiving a daily basis and you know in the email where the amount is written where uh the total amount is written where the name is written where the credit and debits are happening in this particular email body that is you know a structured format which can be trained to the supervised supervised model but what about the unsupervised learning algorithms you know in which uh you know maybe you know you're not happy with some of the organization you're not happy with some of the services let's say an example of any xyz e-commerce company you write some email so you know on a daily basis how many emails they are receiving has a negative of positive feedback millions of emails and they cannot even you know hire someone so many resources

33:12 so these models are helping for you know just taking these particular emails reading one by one analyzing what exactly the problem they are facing and yeah how many people are basically in angry mode how many people are you know giving a good results based out of which they are able to change the strategy as well like they have on what part they have to enhance for the organization so you know basically whatever you can done i mean you can do for multiple packages sir uh the only feature of a scientific kit you know here is uh just because you have to extract some of the details from any of the unsupervised model you will be able to do from this particular library and also you know if i want to define something

33:58 so if you know your this sqlearn library contains a lot of efficient tools or machine learning and including classification aggression clustering and dimensionally reduction you know you want to do all those things you then you can use all those uh this particular library so majorly i mean which is already written you you can use for the cross validation what is cross validation so basically there are various methods to check the accuracy of supervised models or unseen data that can be done from here you know um so if you want to check acquainted what is accuracy basically in terms of i spoke about some example of the email so if you're extracting the data you know amount uh name the last stream all those things can be done from this particular

34:43 cross validation itself it's a kind of you know validation and another thing is peer validation so that is done by this particular model library then we have you know a feature extraction i was telling you know that can be done features from the images even you could do from you know from the words you can extract even from the images also you can extract some of the features so somebody has the box so you know i'll give you the best example you know there sometime it happens you receive some of the delivery box from amazon or any other big ecommerce site and you the product is different from whatever you ordered when you uh give the image to them and they evaluate those images so that is all done by this one

35:29 particular libraries only and 70 times the data is activated and they're able to replace or return whatever items they want to do based on your need and you know there are multiple data sets as well they came in handy while learning sky kit learn i i had uh you know there were some sas academic data sets there were books on house i'll i'll give you an example of boston housekeep uh by doing the coding itself i have the ready model itself right now and having them handy by learning these libraries and uh when i was talking about the unsupervised i just wanted to add one more thing you know and the unsupervised uh all things can be happen like you know starting off clustering factor analysis principle component analysis all

36:14 supervised unsupervised can be done through this particular kit then we have numpy numpy is a library in again used for scientific mathematical data but uh only point to be noted in numpy is you know numpy is only used for mathematical data so no objective data is being used for you know in case of numpy it supports uh you know 2d 3d uh kind of data 2d is something you know there is multi-dimensional data which has been handled by this particular numpy but for an example i can give you the climate the stock data exchange data predictive analysis from past 10 years

37:00 that can be done through the numpy itself so basically numpy stands for numerical python is a library consisting of multi-dimensional data objects and collection of routines for processing those areas and you know using numpy mathematical and logical operations on areas can be performed by using only a single method so you have to just you know learn like you have to do some you write some you have to do multiply right you know you have to write cross or something there are some methods which is readily you can be used easily for these statistical uh manipulations and uh i think that that's it that's more or less you know i mean all mathematical operations can be done where whatever you can yeah you know see whatever you you can see like you

37:46 know if there is four sets of data you can count right if whatever you can count whatever you can um you know you can do some sigma operations uh square root all those things which you can do and give some get some results it all can be done to the numpy itself then we have the last library introduction that is uh theano so theano is also a python library that allows us to evaluate mathematical operations including multi-dimensional areas uh so efficiently and you know it's very easy to install the stride pip install thiano you will be able to able to install this particular library and why thiano has been used you know i mean there are some other points which have already written tight integration with numpy transparent use of gpu

38:32 you know basically i'll tell you look i mean if you want some stable optimization of your solution you know you can find out some unstable expressions and can be used in a more stable form to evaluate them basically you know you're getting some data from different different vendors so let's say example i'm i'm right now doing invoice analysis from different different companies uh i'm writing customer id the other vendor is writing id the third person writing employee id so if you want to do stability optimization then you can use theano as a python library and then you know uh if you want to execute so as mentioned earlier like theano uh you know we can so most recent gpus which are using so if you want to execute the expression in your cpu or gpu right it is much faster

39:19 you know than python and the symbolic differentiation is there so it's smart enough to automatically create symbolic graph for computing gradient c venue i'll tell you order the gradient they have a subset of examples where what are exactly the gradients and so you know they basically theon is a python library that allows you to define optimize and evaluate mathematical expression involving multi-dimensional arrays efficiently some some of the theano example you know i'll tell you in the later stage of this particular these are all you know libraries i'll tell you like chaos is the same same you know having some additional features on this you know if you wanted to do some you know some graphical representation

40:06 uh you want to visualize something you know megan you can um show like what x axis is why what y axis is and you know it's a it it's also handles for the api so you know high level api handles to be making models defining layers or set of multiple input output models so ekas also compiles a model with loss and optimizer functions so what is loss and optimizer function which i'm coding right now that's you know so every model right now you know whatever whatever you predict so there is some lost percentage even if you take an example you know if you're some computer background so whenever the data transfers to one place to another place from one server to another client system there's some packet loss so we always

40:51 create some loss percentage as well so you know no data can be equate that is you know all those uh you see all the standards never we we never say hundred percent of data was accurate because there is some error which is already you know in all the systems maybe some integration system maybe some of the other ethic which has not been followed so that is the last percentage we we we keep you know whenever we have to analyze the data we have to train the model then you have to keep this loss and optimizer function so loss i told you another is you know optimizer optimizer is something you know we have some of the algorithms by which you can optimize your solution you need not to write that particular algorithm optimizer there are some optimizers based on your your model you have to use

41:36 those optimizers then we have computational graphs which which make tensors or variables because it has been handled in a back-end engine as well so basically what happens is you know if you so you know we know the management does not have time to see all those or data whatever you are giving to them they wanted to see what exactly how the graph is performing how we can take the action steps so this is a you know kind of a differentiation which asks me there is another is mad plot live as well which is doing this similar kind of thing but this is having a you know high resilient you know how python is growing day by day so there is a lot of introduction which is having so need not to worry about these libraries uh there is only thing is you know we have to just uh see how the function has been used

42:22 already algorithms people have already written no need to worry about those things no inventions needed here only things you need to write you need to just uh uh expose you know you just to call those functions you know and um i'll tell you one of the you know some of the differences between also from what is chaos and tensorflow so you know um the purpose of chaos is rapid deployment for making model with standard layers you know um and there if you wanted to use tensorflow then you know it allows you to make an arbitrary computational graphs or model layers so if basically chaos gives you exact what exactly you know the methods you can apply here which tensorflow doesn't all right then we

43:08 have our last library which is nltk then we switch on to the coding part nltk is a study of natural language text text analysis uh sentiment analysis i guess already covered this where again you know i'll take an example you know of twitter you know how twitter uh i mean if you guys think how twitter is removing a lot of posts on a daily basis you know daily basis 300 000 if you calculate you know 300 000 of posts are being deleted from twitter uh just because of this particular technology if you're writing some you know some terrorist based uh what we say some post is there or else something related to religious which is not

43:54 acceptable with any of the government so this is a kind of technology which is helping a lot even there we are able to remove the images as well which is not fruitful for any of the social networking websites so nltk is a leading platform for building python programs to work with human language data it provides easy to use interfaces and you know if you wanted to uh such as you know word net along with suit for text processing libraries classification tokenization and parsing the data getting the resultant of the data how you know i told you the example of how people reaction is if you launch some product in the product in the market and you launch the website as well you know they are writing something that's fine i

44:39 mean that is fine completely fine they're writing something you know maybe some other language so what nltk does is it creates one platform creates into one language their language their computer language that is bytes language and then they're able to judge the customer is you know happy with this product which we have launched or else we have to improvise in any of the sector so nltk is has been called as a wonderful tool for teaching and working in computational linguistics using python and an amazing library to play with the natural language so that's it you know from the annual tk site all right uh let's get started the screen which you can see right now that is of uh anaconda anaconda is basically uh just for

45:26 beginners anaconda is basically a platform a framework on which the python is deployed a python code you can write and uh it's extremely easy to download anaconda just go to google and uh click on anaconda download whatever system you are having mac or desktop as a windows or windows 10 you can download it and you know once you download there is a other option called as jupiter lab you have to click on jupiter i'll show you like how you can see the screen so once you download this anaconda navigator you will be able to see jupiter lab notebook so i mean for this particular purpose we will be going jupiter lab that is an extensive environment in which you know we write all python code in which you know even

46:12 all those uh libraries which we have discussed right now uh that can be imported and you have to just download and install it once you install this particular package you have to just uh you know go and what depending on your need you keep on installing and then you can use depending upon your demands okay so okay uh we'll go through this particular model you know that is uh minutes data set so basically minist uh what are the case study basically you know ministry uh menace database is a database in which 60 000 training images uh have been you know calculated in which you know they're they're where and we have taking 10 000 testing images we are taking 10 000 testing images and you know our case study is to

47:00 allocate one image type with one label so you know that there's a kind of um repository the case study which you're building so just to give you a background basically you know the minist which is the word which you which you can see even you can search it on the google as well that's you know that's a menace database that's uh already uh handwritten digits available from this page and uh has a training set of 60 000 examples so basically you know the database is already ready with us we are just doing on this particular model to will train 10 000 models automatically it will be reflected to a next 50 000 model so that's how idea is to give one-on-one mapping to one image to one label so let's say so i'll show you like that

47:46 so if you see this particular image that has been mapped for 8 000 if you see 8 000 here 8 000 have got this particular image so now this particular image is allocated to 8 000 only we'll go one by one how we have done achieved this particular model and how the ai model has been trained so that you know each and every number will be having a different image and how what accuracy we are getting so okay so um we go one by one one is you know we have first of all getting a tensorflow version uh we have two point ticks that is there was one version for 1.6 2.6 there is no no you know very uh great difference between 1.6 and 2.8 that's uh the major thing which they have achieved in

48:32 tensorflow two point x's if you can see tensorflow.kegas uh that is a kind of uh you know the inter coupling or tightly coupled they have done for two particular packets you know there's one library another library they have packaged all together both okay we'll go one by one first of all we have imported tensorflow and uh whatever version you're using just write the version here you'll be getting output as 2.3 just for our sake you know which version you are using first and then you have we have initialized a random number generator so basically you know we have written random dot seed zero what a seed i'll make you understand what a seed the seed is the starting point of for the sequence and guarantee is that if you

49:20 start from the same seed you'll get the same sequence of numbers so you know if you see random you know if we do not define this then what's gonna happen you know uh if you run the same code on your machine there will be some different uh type of models you know to set the standard we have to define like for each maintaining the consistency for each of the machines wherever the code will be deployed will be having the similar kind of results we have to define c dot zero then we have you know ignore the warnings that is you know we are ignoring the warnings maybe you know there are multiple warnings which you will be receiving that we are currently ignoring because we do not want this whenever you load the data there are multiple pixels going on you know you now be analyzing

50:06 close to 10 000 testing images and having a lot of images pixels and you know we want to ignore this that's why we are using this let's load that data set so first what we're doing is um we are you know the syntax first of all you know tensorflow chaos or data sets why we're using tensorflow and chaos is basically you know chaos is uh for this particular model we need both the packages together so that's what we have used tensor flow dot chaos or data set data set is nothing but you know data tables because now there will be pixel formulation there will be a set of basically tables which will be used to manipulate the data and to assign each data to one particular image

50:51 so one is you know i mean this is a form which we have two guys x train and y train uh so what exactly it is x train and x uh x test is basically uh a gray scale of rgb codes from ranging from zero to 255 and where y train and y test contains the labels from zero to nine so we have ten thousand records but you know in binary if you see there is a combination nine thousand sixty thousand forty thousand that will be in between zero to nine only so that's what we have to use and then we have what we have done is we have loaded the data so there's a minus data which we already defined and we have to use a function which is already present in tensorflow library this is load data we're loading all the data there then we have uh

51:38 you know we have to visualize later so as i mentioned in this session as well like there is a library which is called as matplotlib and dot pi plot is basically you know library function is there which we have using here to visualize the data you know if you see this image this if you do not use this uh matte plot lip you will not be able to see this particular image and then we have you know we are just printing so print is nothing it's you know whatever you see here you know matplotlib.image xmas at this particular uh grayscale number they have given that is coming from the print command basically and then they are plt img show x train and we have grey scale we went to the gray model here so that is a black and white

52:24 combination that is how we are giving cmap as gray so this is basically a model which you can see image you can see which we have defined that 8 000 we will be able to see for eight thousand number we will be able to see this particular image now what we are doing is uh if you i mean just notice this you know mean how x axis and y axis is there if you can see uh the y-axis is starting from 25 to zero and x-axis is starting from zero to 25 i'll tell you what the significance for this is uh in the latest stage now we are only printing to showcase you know what exactly we have so what we have done is you know print exchange so how many models we have to train that is 60 000 you know and that shape shape is 28 comma 28

53:10 what is this 28 comma 28 is basically you know if you can see um the image coordinates can be only till 28 to 28 00 to 20 that's how we have given the uh range to this particular model and then going forward now we have reshaping the training test so what we're doing currently i mean as a layman language because there are multiple heterogeneous participants i'll tell you like what we're doing is you know we are playing this particular model which we have defined now you know all the possibility which we'll be defining here like what can be happened there can be one more model there can be 3d model there can be three more parameters all those things will be able to train them so then we are reshaping and training

53:56 the test sets what we have that is a sequential model of tensorflow.chaos expects data in format of n underscore e and underscore h w and c what is parameters are e is basically examples which we are feeding right now what we have done is we are given the example so this is we are feeding to that particular model h is height height is 28 if you can see and width is also 28 that is also a bit in the graphical model and sees number of channels you know number of channels we have defined as one because you know there is only two channels that are zero and one right now we are given one as a channel so we have enhanced one more capability of this particular model then the third step is i mean the what

54:42 we are doing one by one you know we should see all the models you know what we are doing is first step in any of the data whatever you get that will be printing you know shape of the data reshaping training and test data same things which you have to use i mean do not worry about these kind of syntax the same sequence will be used across all the models first time i know it will be difficult to understand but you know if you go through this particular code which you find very easy you know when you go through one or you know five six models what that then you go for uh normalizing the data where what we're doing is basically you know uh we know so normalization in database what it is called you know normalizing is you know uh we are dividing the data into one by one you know it's like you know uh how possibility of the data can

55:28 be so that you know each data frame can be divided into one module so what we are doing here is we have given you know uh that the train uh right now we are given defined is float32 why we are doing float32 because whenever we divide some number by 255 though there can be a decimal data so decimal data has to be stored in some different type so that's why we have converted x dot train as type to float 32 so that now in a future stage what we are doing is we are dividing some of the arbitrary number that is 255 from z ranging from 0 to 55 so there can be no error in case we are giving float32 if this is having string or integer it will

56:13 definitely give an error then going in the next day that is a pin shape of data and number of images so uh here what we're doing is you know i mean we're giving the elaboration of all those data you know i mean what exactly we have the data so these are all print statements you know need uh so basically these are the statement which i have written just to make you understand there is no significance of this particular print statement this is just for us to understand what is the data inside which variable so x train type are having shape this is a particular data 60 000 records 28.28 comma 1 which i have shown you earlier then we have how many data sets we have to train that the 60 000 what is the

57:00 test data on which data we will be doing the testing that is 10 000 what is the maximum value and minimum value maximum value is 1.0 and minimum value is 0.2 i told you like the channels channels is only two type that is zero and one so maximum is x strain is one and minimum is one zero point zero then we are going for the vector analysis here we are categorizing basically on the basis of the shapes so um the syntax for this is again you know we have to use a util class so first of all we are converting the class vectors to binary because you know computers only under able to understand the binary methodologies so we this is an important step that is a third step of any of the modeling techniques

57:45 and how many uh number of classes we have taken as 10 and then we are doing the categorical cross entropy on loss so loss percentage i already told you the example by uh always in any of the model we used to engulf the loss as well you know we uh whatever i mean it's not like you know machine learning model you train today and you know all things will be working by tomorrow there will be some losses which will be uh losses means there will be some inaccuracy in the data which will be i mean you you make your model more robust you train your model more with more and more data the that's how the loss uh percentage will be going in a decreasing manner after some time then uh going forward we have you know

58:30 initializing the sequential model and we again you know we have to just model how i mean how do we want actually you know i mean there's a model which we have to train right so there are three layers which we have said one is sequential another is we will be flatting flattening the data that is you know in a 2d or one area before we building the fully connected layers then at the last step we'll be adding two dense layers that is dense layers you know major thing is we experiment of any of the number of neurons for the first dense leg however the final dense layer must be having nuance equal to the number of output classes because this works and this is how which we are doing you know we are again taking uh the model so if you have noticed you know we uh after

59:16 tensorflow.js somewhere we are taking the utils somewhere taking your model somewhere we are taking a layer so that is depending upon your lead so basically you know inside the model package there is a uh you know inside this is only we are able to use inside layers we will be able to provide things you know if you will not write layers here you will not be able to get a intelligence for this then said you will be getting error for this particular thing so going forward here we have just compiling this particular model whatever we have done and after compiling this model all the batches will be getting started and you know you have to train the batches we are taking the batch size at 32. epochs we have taking this 10 what is it basically epochs epochs is a term used in machine

1:00:02 learning and indicates a number of passes of the entire training data set the machine learning algorithm has completed so that's how you know it's defined as epochs these are the same same terms you know i mean we will understand like uh when you write fit in your code you write the bracket will be automatically able to get in how many parameters you have to pass and depending upon that parameters you have to give all those uh size and depending upon your need and if the batch size is the whole training data set then the number of epochs is the number of the iterations so right now we're taking only 10 but the batch size if you calculate 60 thousand or ten thousand or twenty thousand you have to give all because you know epoch is something iteration for loop iteration if you say

1:00:48 uh one match with other ten second match with other ten you know it's a kind of method so right now for you know for the data accuracy we are using epochs in here and then after calculating this particular you know we will be running this particular model you will be able to see how the model epochs have been calculated for 10 uh particularly methods if you observe here the loss percentage uh you know is decreasing you know by line by line if you see the loss percentage at the last line that is coming out to be only 0.16 so the accuracy of the data is 95 so that is how you know uh the epoch plays a role here and then once we evaluate so when when

1:01:34 we we evaluate and see what is the accuracy of the result it shows how you have fed 313 models and out of that 313 models have now the data which is accurate without having any kind of losses and accuracy is 95 without the with the loss of 16 you know we have only uh given 313 models and you have to you are able to get accurate results now you will be engulfing this particular data with 60 000 records you will be able to map one-on-one relationship with all the models you know whatever models you are having so this is how you mean uh how the machine learning model any machine learning model works you know so major things is you have to uh make you make yourself understand you know how it

1:02:20 is starts basically you know you reshape you train that is you know the pre-processing of the data how you can pre-process your data you normalize your data you segregate the data then you see like how the data is looking like on what places we can analyze data what shape we can do what you know category how we can define the category so see there can be a different different type of invoices one can be invoices of you know po typo invoice there can be invoice of some luggage they uh you know some of the products which you have bought from the supermarket they can be multiple type of invoices you know so it's important to first normalize any data and second point the point is categorize the data you know if you are not categorizing the data you will not be

1:03:06 getting the results appropriate then you know identifying uh which model you have to use you have to process the data sequentially or you have to process parallely which model you have to use what kind of uh etiquettes which you have to follow you know what kind of layers which you have to you know every so see i mean you you can study these layers basically you know if you um then say something you know if you uh these are basically layers by which the machine learning model has already been trained and you will be able to feed the data you will be able to achieve the results on those basis that's it so uh this is all uh from my site for today's session thank you so much everyone for joining

1:05:47 you