__ Shirley 2021-08-10 11:37:59 阅读数:809

本文一共[544]字，预计阅读时长:1分钟~

serial
tutorial
rt
thread
intelligent

** Preface **

This document is very long, so we will divide it into 5 A series of articles , Because machine learning is not pure software development , Simply call the library function API, It needs some theoretical support , If the theoretical part is not introduced at all , Maybe I don't know why the model is designed like this , There is something wrong with the model . However, if the document is too long, it may be difficult for you to read it patiently , Especially in the theoretical part, there will be many formulas , But machine learning is right ** Theoretical basis ** and ** Programming ability ** There are some requirements , I believe that there will be a lot of gains if we stick to it , I also try my best to introduce the theory and application .

This article assumes that everyone already knows how to use RT-Thread Of env Tools download packages , And generate project upload firmware to stm32 On , Because these two serials focus on Loading onnx General machine learning model , About RT-Thread You can go to the official website documentation center ：https://www.rt-thread.org/document/site/ Look for .

First , Briefly introduce the scope of each topic mentioned above (Domain), Artificial intelligence (Artifitial Intelligence) It's the biggest topic , If you use a picture to illustrate ：

Then machine learning (Machine Learning) That's the theme of this document , however machine learning It's still a very big topic ： Here is a brief introduction to the three types mentioned above ：

** Supervised learning (Supervised Learning):** This should be the most widely used area , For example, face recognition , I'll give you a lot of pictures in advance , Then tell you which of them contain faces , Which does not contain , You summed up the features of the face from the photos I gave , This is the training process . Finally, I will provide some pictures that I have never seen before , If the algorithm is well trained , It can distinguish whether a picture contains face or not . So the biggest characteristic of supervised learning is the training set , Tell the model what's right , What is wrong .

** Unsupervised learning (Unsupervised Learning):** For example, online shopping recommendation system , The model will classify my browsing records , Then automatically recommend relevant products to me . The biggest characteristic of unsupervised learning is that there is no standard answer , For example, water cups can be classified as daily necessities , It can also be classified as a gift , No problem .

** Reinforcement learning (Reinforcement Learnong): ** Reinforcement learning should be the most attractive part of machine learning , for example Gym There are many examples of training computers to play games by themselves and get high scores . Strengthening learning is mainly through trial and error (Action), Find a way to get the most out of yourself , That's why a lot of examples are computer games .

** classification (Classification): ** For example, handwriting recognition , The characteristic of this kind of problem is that the final result is discrete , The final number of categories can only be 0, 1, 2, 3 Not really. 1.414, 1.732 Such decimals .

** Return to (Regression): ** For example, the classic house price forecast , The results of such problems are continuous , For example, house prices will change continuously , There are infinite possibilities , Unlike handwriting recognition, it's just 0-9 this 10 Species category .

** Artificial neural network (Artifitial Neural Network):** This is a more general method , It can be used in various fields to do data fitting , But there are more suitable algorithms for image and voice .

** Convolutional neural networks (Convolutional Neural Network):** Mainly used in image field , It will also be introduced in detail later .

** Cyclic neural network (Recurrent Neural Network):** The comparison applies to sequential input like sound , So there are many applications in the field of language recognition .

So to conclude , This document describes Artificial intelligence The following development is relatively fast machine learning Branch , Then the solution is machine learning Supervised learning Below Classification problem , It's using neural network Inside Convolutional neural networks (CNN) Method .

1 Neural network theory

This part mainly introduces the whole running process of neural network , How to prepare the training set , What is training , Why train , How to train , And what I got after training .

In order to make the above model fit these scattered points well , Our goal is to change the model parameters θ0 and θ1, That's the slope and intercept of this line , So that it can respond to the trend of scatter point well , The following animation is a very intuitive reflection of the training process .

You can see , It started as a nearly horizontal line , But slowly its slope and intercept move to a better position , So here comes the question , How can we evaluate whether the current position of this straight line can meet our needs ？ A very direct idea is to find out the actual value of all scattered points y And the test value of our model h θ The absolute value of the difference , This evaluation index is called loss function J(θ) (cost function) ： The right side of the function is divided by 2 It's for the convenience of counting down , Because if the formula on the right is derivative , The square above will get one 2, It happens to be in the denominator 2 Offset . In this way, we have the evaluation indicators , The smaller the loss function, the better , In this way, we can know that the current model is incorrect yes Can meet the needs well , The next step is to tell the model how to optimize in a better direction , This is training (Training) The process .This kind of training method is very famous Gradient descent method (Gradient Descent), Of course, there are many improved training methods such as Adam, In fact, the principles are almost the same , I won't give you too much introduction here .

The process of machine learning is summed up as , Let's design a model first , Then define an evaluation index called loss function , So we know how to judge the model , The next step is to use a training method , Let the parameters of the model move in the direction that can reduce the loss function , When the loss function is almost no longer reduced , We can think that training is over . The final training is the parameters of the model , With the trained model, we can predict other data . By the way , The above linear regression actually has a standard theoretical solution , That is to say, there is no need to go through the training process , One step to get the optimal weight , We call it Normal Equation： that , There is a theoretical solution in place , Why do we need to train step by step ？ Because the formula above has inverse operation of matrix , When the matrix size is small , The amount of matrix inversion is not large , But once the scale of the matrix goes up , It is almost impossible to use the existing computing power to find the inverse , So this time we need to use gradient descent training method to approach the optimal solution step by step .

Let's go back to the handwriting recognition example , The linear regression described above results in a continuous number , But the final goal of handwriting recognition is to get a discrete value , That is to say 0-9, So how can this be done ？

This is the model of the previous part , It's very simple , Just add one more... To the final result sigmoid function , Limit the final result to 0-1 That's all right. . Just like the formula in the picture above ,sigmoid Function is ： If we apply it to the linear regression model , We get a nonlinear regression model , That is to say Logistic Regression： In this way, we can make sure that the final result we get is 0-1 Between , Then we can define if the final result is greater than 0.5 Namely 1, Less than 0.5 Namely 0, So a continuous output is discrete .If we don't get the results so quickly , But insert another layer in the middle ？ So we have a neural network with a hidden layer .

In the picture above , We use it a representative Activation function (activation function) Output , The activation function is also mentioned in the previous section sigmoid function , To limit the output to 0-1, If not , It is likely to go through several layers of neural network calculation , The output value will explode to a very large number . Except, of course, sigmoid Out of function , There are many more activation functions , For example, the next part is very commonly used in convolutional neural networks Relu. in addition , We use bracketed numbers to represent the number of layers of neural network . for example a (1) Represents the output of the first layer neural network . Of course , The first layer is the input layer , It doesn't have to go through any calculations , So you can see a (1) =x, The activation function output of the first layer is our input directly x. however ,θ (1) It's not a parameter representing the first level , It's a parameter between the first layer and the second layer , After all, parameters exist in the calculation process between two-layer networks . therefore , We can summarize the above neural network structure ： ● Input layer ：a (1) =x● Hidden layer ：a (2)= g(θ (1) a (1) )

● Output layer ：h (θ) =g(θ (2) a (2) ) If we set the last output layer node to be 10 individual , Then it can be used to express 0-9 this 10 A number . If we add more hidden layers , Doesn't it look like interconnected neurons ？ If we go deeper Go Deeper ( The author mentioned in the paper that , His inspiration for deep learning actually comes from inception )

So we have a deep neural network ：

If you want to know , How many hidden layers should be selected , Each hidden layer should select several nodes , This is where you come from , Where to go is the same , It's the ultimate problem of neural networks ???? Last , The training method of neural network is used Back propagation (Back Propagation), If you are interested, you can find a more detailed introduction here .

- Cov2D
- Maxpooling
- Relu
- Dropout
- Flatten
- Dense
- Softmax

Next, we will introduce these operators one by one .

Here is why convolution is introduced , Although the previous matrix multiplication can solve many problems , But when it comes to images , To a 1920*1080 Multiply the image , It's just one. [1, 2,073,600] The matrix of the , The amount of computation is not small , And convolution , The amount of calculation will be greatly reduced ; On the other hand , If we compress a two-dimensional image into a one-dimensional vector , In fact, the information that pixel points are related to each other in the up, down, left and right directions is lost , For example, a pixel is usually similar to the surrounding color , This information is very important image information many times .

The advantages of convolution operation are introduced , So what exactly is convolution ？ In fact, convolution is a simple addition, subtraction, multiplication and division , We need an image , And then there's a convolution kernel (Kernel)：

The image above goes through a 3x3 Convolution kernel operation , The edge of the image is extracted very well , The following animation clearly introduces matrix operation ：The convolution kernel used in the above animation is a 3x3 Matrix ：

If we pause the animation ： It can be seen that the convolution operation is actually to scan the convolution kernel on the image according to the row and column , Multiply the numbers of the corresponding positions , Then sum it , For example, the convolution result in the upper left corner 4 That's how it's calculated ( Here we use ∗ For convolution )： Of course, the above calculation process is not rigorous with equal sign connection , However, it is convenient to explain the calculation process of convolution . You can see , The computation of convolution is very small compared with the fully connected neural network , Moreover, the relevance of the image in two-dimensional space is preserved , So there are many applications in the field of image . Convolution is very easy to use , But after convolution, the image size becomes smaller , For example, above 5x5 The matrix goes through a 3x3 The result of convolution kernel operation is 3x3 Matrix , So sometimes in order to keep the image size constant , It will be used around the image 0 fill , This operation is called padding. however padding There's no way to keep the image size the same , Because the convolution kernel of the above animation only moves one lattice in one direction at a time , If every time you exercise 2 grid , that 5x5 The image of 3x3 The convolution of becomes 2x2 The matrix of the , The number of steps each convolution kernel moves is called stride. The following is the image size calculation formula after convolution operation ： For example, the width of the image above W = 5, Convolution kernel size F = 3, Not used padding therefore P = 0, Steps per move S = 1：Here's an explanation , All the above calculations are for a convolution kernel , In fact, a convolution layer may have multiple convolution kernels , And actually a lot of CNN The model is also convolution kernel with the number of layers later , More and more .

The problem of model over fitting has not been mentioned before , Because the neural network model in the training process , It is likely that the model fits the training set provided by itself very well , But once you come across data that you haven't seen , We can't predict the right result at all , That's when there's over fitting .

that , How to solve the over fitting problem ？Dropout It's a very simple and crude way , From the parameters that have been trained , Randomly pick out some and discard them and reset them to 0, That's why its name is Dropout, Just throw away some parameters at random . It's an incredibly simple way , But it's surprisingly easy to use , For example, just in maxpooling Then randomly discard 60% Trained parameters , We can solve the over fitting problem well .Flatten It's just like the literal meaning , Put one 2 Flatten a matrix of dimensions , For example, such a matrix ：

It's that simple ...In the first part, the operators related to convolutional neural network are finally introduced , The second part will introduce how to use Keras (Tensorflow) Machine learning framework trains a handwriting recognition model , Finally, the third part introduces how to import the generated model into stm32 Run above .

** Stanford classic machine learning video **

link ：https://www.coursera.org/learn/machine-learning

** Linear regression **

link ：

https://towardsdatascience.com/introduction-to-linear-regression-and-polynomial-regression-f8adc96f31cb

**END**

**RT-Thread**

Make the development of Internet of things terminals simple 、 Fast , The value of chips is maximized .Apache2.0 agreement , It can be used in commercial products free of charge , No need to publish the source code , No potential business risks .

Long press QR code , Pay attention to our

版权声明：本文为[__ Shirley]所创，转载请带上原文链接，感谢。 https://car.inotgo.com/2021/08/20210810113604367G.html

- "Xie Dajiao" Yu Yuexian died in a car accident, and their lives were also terminated by the accident
- At the age of 44, Chen Shu appeared in the brand store to shop, bought three carts of clothes, and his body size was exposed
- Buy Volvo to avoid this car. It is small and expensive. The sales volume in June was only 350 units!
- "Xie Dajiao" died in a car accident in Yuexian. He was only 50 years old. Friends in the circle sent messages to mourn
- Police responded to the cause and details of Yu Yuexian's car accident: driving into a camel in the early morning, and the driver and others were injured
- Actor Yu Yuexian died in a car accident at the age of 50. The police informed Yu Yuexian of the cause of the car accident
- “谢大脚”车祸去世，小沈阳姜超发微博悼念，翟天临都“很难过”
- 被惋惜的不止“谢大脚”，这3位车祸去世的明星，每位都让人心痛
- 警方通报于月仙车祸去世：小型客车发生交通事故造成于某某死亡
- 50岁于月仙车祸去世，与老公结婚20多年无子！把患病弟弟当儿子养
- "Xie Dajiao" died in a car accident. Xiao Shenyang Jiang Chao sent a microblog to mourn. Zhai Tianlin was "very sad"
- It is not just "Xie Dajiao" who is regretted. Each of the three stars who died in a car accident is heartbreaking
- The police reported that Yu Yuexian died in a car accident: a traffic accident on a minibus resulted in Yu's death
- Yu Yuexian, 50, died in a car accident and married her husband for more than 20 years without children! Raise your sick brother as a son
- 男星开百万豪车太高调，多次压线并占用应急车道，网友痛批没素质
- 于月仙车祸去世，《乡村爱情》官博发文：“仙”人已逝，如“月”婵娟
- The male star drove a million luxury cars too high-profile, pressed the line for many times and occupied the emergency lane. Netizens criticized him for lacking quality
- Yu Yuexian died in a car accident. The official blog of rural love wrote: "the immortal" is dead, such as the "Moon" Chan Juan
- 现在上车以太坊挖矿还有戏吗
- Now get on the bus and dig in Ethereum. Is there any play
- 于月仙车祸细节曝光，凌晨三点去参加活动，与丈夫张学松无儿无女
- The details of Yu Yuexian's car accident were exposed. She went to the event at 3 a.m. and had no children with her husband Zhang Xuesong
- 演员于月仙在阿拉善因发生车祸不幸去世
- Actor Yu Yuexian died in a car accident in Alashan
- 于月仙车祸前，在《乡村爱情》中已被边缘化，谢大脚成了广告符号
- 于月仙车祸去世：再见了，《乡村爱情》里的“小卖部女神”
- “谢大脚”于月仙车祸去世，众星发文悼念，章子怡说的话令人泪目
- Before Yu Yuexian's car accident, she was marginalized in rural love, and Xie Dajiao became an advertising symbol
- Yu Yuexian died in a car accident: goodbye, the "canteen goddess" in rural love
- 50岁“谢大脚”车祸去世，《乡村爱情》15年，已经有5位演员走了
- 50岁于月仙车祸去世！途中和两只骆驼相撞，最后一条动态令人泪目
- 于月仙车祸去世，《乡村爱情》再无“谢大脚”，永远怀念！
- "Xie Dajiao" Yu Yuexian died in a car accident. The stars sent a document to mourn. Zhang Ziyi's words made people cry
- At the age of 50, "Xie Dajiao" died in a car accident. Five actors have left in "country love" for 15 years
- 一路走好！50岁“谢大脚”于月仙车祸去世，与老公结婚多年无儿女
- 警方通报于月仙车祸原因：与两头骆驼相撞，司机和其他成员只受伤。“乡爱”官微发文悼念
- Yu Yuexian, 50, died in a car accident! On the way, he collided with two camels, and the last one was tearful
- Yu Yuexian died in a car accident. There is no "Xie Dajiao" in rural love. I will always miss it!
- All the way! Xie Dajiao, 50, died in a car accident in Yuexian. He married his husband for many years without children
- The police informed Yu Yuexian of the cause of the accident: the driver and other members were only injured when they collided with two camels“ "Township love" official wechat sent a document to mourn
- 于月仙因车祸去世，长贵可以复活，但《乡村爱情》从此再无谢大脚
- 50岁于月仙车祸去世，《乡爱》再无谢大脚，小沈阳：小姨一路走好
- 于月仙车祸细节：被撞两只骆驼也死了，全车4人，只有她去世
- 赵本山小姨子车祸去世，《乡村爱情》剧组悼念，50岁一辈子未生子
- 毫无底线！于月仙车祸去世，网红直播谢大脚超市引众怒
- 50岁于月仙遭遇车祸突然离世：原来人生真的没有那么多来日方长
- Yu Yuexian died in a car accident, and Changgui can come back to life, but "country love" has never had Xie Dajiao again
- Yu Yuexian died in a car accident at the age of 50. There is no thanks for big feet in rural love. Little Shenyang: let's go all the way
- 警方通报于月仙车祸原因：与两只骆驼相撞！赵本山女儿发文悼念
- 蹭热度无底线？于月仙车祸去世，多位网红大脚超市直播被网友骂惨
- Details of Yu Yuexian's car accident: two camels were also killed. There were 4 people in the car, and only she died
- Zhao Benshan's sister-in-law died in a car accident. The crew of "rural love" mourned that she had no children at the age of 50
- No bottom line! Yu Yuexian died in a car accident. Xie Dajiao supermarket attracted public anger
- Yu Yuexian, 50, died suddenly in a car accident: it turns out that life really doesn't have so much time
- Police informed Yu Yuexian of the cause of the accident: collision with two camels! Zhao Benshan's daughter sent a document to mourn
- No bottom line for rubbing heat? Yu Yuexian died in a car accident. Many netizens scolded her for live broadcast in wanghongdajiao supermarket
- 《乡村爱情》谢大脚扮演者车祸去世：生命来来往往，来日并不方长
- Xie Dajiao's actor in rural love died in a car accident: life comes and goes, and the future is not long
- 著名演员于月仙：因车祸猝然离世，将悲痛和心碎留给丈夫
- Famous actor Yu Yuexian: died suddenly in a car accident, leaving grief and heartbreak to her husband