We can do this easily by adding new Dropout layers between the Embedding and LSTM layers and the LSTM and Dense output layers. Lines 1-6, represents the various Keras library functions that will be utilised in order to construct our RNN. This flag is used for when you're continuing on to another recurrent layer. I've been working with a recurrent neural network implementation with the Keras framework and, when building the model i've had some problems. It can be used for stock market predictions , weather predictions , … Don't worry if you don't fully understand what all of these do! Feedforward neural networks have been extensively used for system identification of nonlinear dynamical systems and state-space models. So what exactly is Keras? It is difficult to imagine a conventional Deep Neural Network or even a Convolutional Neural Network could do this. Recurrent neural networks are deep learning models that are typically used to solve time series problems. We then implement for variable sized inputs. In this part we're going to be covering recurrent neural networks. Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras: 2016-10-10: Feedforward NN: Two hidden layers Softmax activation function Model is trained using Stochastic Gradient Descent (SGD) Keras, sklearn.preprocessing, sklearn.cross_validation: Image classification: A simple neural network with Python and Keras: 2016-10-10 Rather than attempting to classify documents based off the occurrence of some word (i.e. This brings us to the concept of Recurrent Neural Networks . In this tutorial you have learned to create, train and test a four-layered recurrent neural network for stock market prediction using Python and Keras. Error on the input data, not enough material to train with, problems with the activation function and even the output looked like an alien jumped out it's spaceship and died on my screen. Then say we have 1 single data output equal to 1, y = ([[0, 1, 0, 0, 0]]). Let's put it this way, it makes programming machine learning algorithms much much easier. Keras Recurrent Neural Network With Python. In other words, the meaning of a sentence changes as it progresses. If the RNN isn't trained properly, capital letters might start popping up in the middle of words, for example "scApes". Chercher les emplois correspondant à Recurrent neural network python keras ou embaucher sur le plus grand marché de freelance au monde avec plus de 19 millions d'emplois. Tagged with keras, neural network, python, rnn, tensorflow. So basically, we're showing the the model each pixel row of the image, in order, and having it make the prediction. We run our loop for a 100 (numberOfCharsToLearn) less as we will be referencing the last 100 as the output chars or the consecutive chars to the input. It currently looks like this: This tutorial will teach you the fundamentals of recurrent neural networks. It needs to be what Keras identifies as input, a certain configuration. A recurrent neural network looks quite similar to a traditional neural network except that a memory-state is added to the neurons. Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Although challenging, the hard work paid off! If we're not careful, that initial signal could dominate everything down the line. Keras is an incredible library: it allows us to build state-of-the-art models in a few lines of understandable Python code. Build a Recurrent Neural Network from Scratch in Python – An Essential Read for Data Scientists. Recurrent Neural Networks RNN / LSTM / GRU are a very popular type of Neural Networks which captures features from time series or sequential data. Not really! The idea of a recurrent neural network is that sequences and order matters. Now let's work on applying an RNN to something simple, then we'll use an RNN on a more realistic use-case. It was written that way to avoid any silly mistakes! Recurrent Neural Networks (RNN / LSTM )with Keras – Python. You can get the text file from here. It is an interesting topic and well worth the time investigating. Well, can we expect a neural network to make sense out of it? Recurrent neural networks are deep learning models that are typically used to solve time series problems. Recurrent Neural Network models can be easily built in a Keras API. In the above diagram, a unit of Recurrent Neural Network, A, which consists of a single layer activation as shown below looks at some input Xt and outputs a value Ht. It performs the output = activation(dot(input, weights) + bias), Dropout: RNNs are very prone to overfitting, this function ensures overfitting remains to a minimum. The same procedure can be followed for a Simple RNN. Now the number is the key and the corresponding character is the value. Line 13 theInputChars stores the first 100 chars and then as the loop iterates, it takes the next 100 and so on... Line 16 theOutputChars stores only 1 char, the next char after the last char in theInputChars, Line 18 the charX list is appended to with 100 integers. If nothing happens, download the GitHub extension for Visual Studio and try again. We start of by importing essential libraries... Line 1, this is the numpy library. There are several applications of RNN. Made perfect sense! Same concept can be extended to text images and even music. They are used in self-driving cars, high-frequency trading algorithms, and other real-world applications. To make it easier for everyone, I'll break up the code into chunks and explain them individually. It does this by selecting random neurons and ignoring them during training, or in other words "dropped-out", np_utils: Specific tools to allow us to correctly process data and form it into the right format. Recurrent neural networks (RNN) are a type of deep learning algorithm. Line 2, 4 are empty lists for storing the formatted data as input, charX and output, y, Line 8 creates a counter for our for loop. Although the X array is of 3 dimensions we omit the "samples dimension" in the LSTM layer because it is accounted for automatically later on. Line 1 this uses the Sequential() import I mentioned earlier. If you have any questions send me a message and I will try my best to reply!!! The 1 only occurs at the position where the ID is true. For example: Yes! The computation to include a memory is simple. This is where the Long Short Term Memory (LSTM) Cell comes in. Keras is a simple-to-use but powerful deep learning library for Python. The RNN can make and update predictions, as expected. Recurrent Neural networks like LSTM generally have the problem of overfitting. We will use python code and the keras library to create this deep learning model. A recurrent neural network (RNN) is a class of artificial neural network where connections between nodes form a directed graph along a sequence. My input will be a section of a play from the playwright genius Shakespeare. Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. They are used in self-driving cars, high-frequency trading algorithms, and other real-world applications. A guide to implementing a Recurrent Neural Network for text generation using Keras in Python. #This get the set of characters used in the data and sorts them, #Total number of characters used in the data, #This allows for characters to be represented by numbers, #How many timesteps e.g how many characters we want to process in one go, #Since our timestep sequence represetns a process for every 100 chars we omit, #the first 100 chars so the loop runs a 100 less or there will be index out of, #This loops through all the characters in the data skipping the first 100, #This one goes from 0-100 so it gets 100 values starting from 0 and stops, #With no ':' you start with 0, and so you get the actual 100th value, #Essentially, the output Chars is the next char in line for those 100 chars in charX, #Appends every 100 chars ids as a list into charX, #For every 100 values there is one y value which is the output, #Len(charX) represents how many of those time steps we have, #The numberOfCharsToLearn is how many character we process, #Our features are set to 1 because in the output we are only predicting 1 char, #This sets it up for us so we can have a categorical(#feature) output format, #Since we know the shape of our Data we can input the timestep and feature data, #The number of timestep sequence are dealt with in the fit function. You signed in with another tab or window. In this part we're going to be covering recurrent neural networks. Each key character is represented by a number. #RNN #LSTM #RecurrentNeuralNetworks #Keras #Python #DeepLearning. This is the LSTM layer which contains 256 LSTM units, with the input shape being input_shape=(numberOfCharsToLearn, features). This post is intended for complete beginners to Keras but does assume a basic background knowledge of RNNs. For example, say we have 5 unique character IDs, [0, 1, 2, 3, 4]. If nothing happens, download Xcode and try again. ... Recurrent neural Networks or RNNs have been very successful and popular in time series data predictions. In this post, you will discover how you can develop LSTM recurrent neural network models for sequence classification problems in Python using the Keras deep learning library. Tensorflow 1.14.0. A one-hot vector is an array of 0s and 1s. I will expand more on these as we go along. My model consists in only three layers: Embeddings, Recurrent and a Dense layer. An LSTM cell looks like: The idea here is that we can have some sort of functions for determining what to forget from previous cells, what to add from the new input data, what to output to new cells, and what to actually pass on to the next layer. The batch size is the how many of our input data set we want evaluated at once. Keras Recurrent Neural Networks For Multivariate Time Series. This is where recurrent neural networks come into play. Line 9 runs the training algorithm. RNNs are also found in programs that require real-time predictions, such as stock market predictors. We have the data represented correctly but still not in the right format, Line 4 shapes the input array into [samples, time-steps, features], required for Keras, Line 8 this converts y into a one-hot vector. Recurrent Neural Network(RNN) are a type of Neural Network where the output from previous step are fed as input to the current step.In traditional neural networks, all the inputs and outputs are independent of each other, but in cases like when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. This allows it to exhibit temporal dynamic behavior for a time sequence. Welcome to part 8 of the Deep Learning with Python, Keras, and Tensorflow series. Each of those integers are IDs of the chars in theInputChars, Line 20 appends an integer ID every iteration to the y list corresponding to the single char in theOutputChars, Are we now ready to put our data through the RNN? You need to have a dataset of atleast 100Kb or bigger for any good result! This essentially initialises the network. Recurrent Neural Network models can be easily built in a Keras API. ... python keras time-series recurrent-neural-network. Welcome to part 7 of the Deep Learning with Python, TensorFlow and Keras tutorial series. Line 4 creates a sorted list of characters used in the text. This tutorial will teach you the fundamentals of recurrent neural networks. If you'd like to know more, check out my original RNN tutorial as well as Understanding LSTM Networks. If you are, then you want to return sequences. It can be used for stock market predictions, weather predictions, word suggestions etc. Now we need to create a dictionary of each character so it can be easily represented. If you're not going to another recurrent-type of layer, then you don't set this to true. This can work, but this means we have a new set of problems: How should we weight incoming new data? In this case we input 128 of examples into the training algorithm then the next 128 and so on.. Line 10, finally once the training is done, we can save the weights, Line 11 this is commented out initially to prevent errors but once we have saved our weights we can comment out Line 9, 10 and uncomment line 11 to load previously trained weights, During training you might see something like this in the Python shell, Once it's done computing all the epoch it will straightaway run the code for generating new text. Create Neural network models in Python and R using Keras and Tensorflow libraries and analyze their results. I have set it to 5 for this tutorial but generally 20 or higher epochs are favourable. Lets get straight into it, this tutorial will walk you through the steps to implement Keras with Python and thus to come up with a generative model. In this tutorial you have learned to create, train and test a four-layered recurrent neural network for stock market prediction using Python and Keras. Now imagine exactly this, but for 100 different examples with a length of numberOfUniqueChars. Confidently practice, discuss and understand Deep Learning concepts. Line 2 creates a dictionary where each character is a key. Before we begin the actual code, we need to get our input data. You'll also build your own recurrent neural network that predicts You can easily create models for other assets by replacing the stock symbol with another stock code. We implement Multi layer RNN, visualize the convergence and results. Finally, we have used this model to make a prediction for the S&P500 stock market index. Our loss function is the "categorical_crossentropy" and the optimizer is "Adam". In this example we try to predict the next digit given a sequence of digits. With a Recurrent Neural Network, your input data is passed into a cell, which, along with outputting the activiation function's output, we take that output and include it as an input back into this cell. Let me open this article with a question – “working love learning we on deep”, did this make any sense to you? Welcome to part 7 of the Deep Learning with Python, TensorFlow and Keras tutorial series. Keras is a simple-to-use but powerful deep learning library for Python. Notice how the 1 only occurs at the position of 1. Follow edited Aug 23 '18 at 19:36. from keras import michael. Finally, we have used this model to make a prediction for the S&P500 stock market index. They attempt to retain some of the importance of sequential data. Recurrent neural Networks or RNNs have been very successful and popular in time series data predictions. So that was all for the generative model. How should we handle the recurring data? Building a Recurrent Neural Network. Framework for building complex recurrent neural networks with Keras Ability to easily iterate over different neural network architectures is key to doing machine learning research. In more technical terms, Keras is a high-level neural network API written in Python. Line 4 we now add our first layer to the empty "template model". Faizan Shaikh, January 28, 2019 . Although we now have our data, before we can input it into an RNN, it needs to be formatted. Whenever I do anything finance-related, I get a lot of people saying they don't understand or don't like finance. The example, we covered in this article is that of semantics. In the next tutorial, we'll instead apply a recurrent neural network to some crypto currency pricing data, which will present a much more significant challenge and be a bit more realistic to your experience when trying to apply an RNN to time-series data. Ability to easily iterate over different neural network architectures is key to doing machine learning research. Not really – read this one – “We love working on deep learning”. Learn more. If for some reason your model prints out blanks or gibberish then you need to train it for longer. We can then take the next 100 char by omitting the first one, Line 10 loops until it's reached 500 and then prints out the generated text by converting the integers back into chars. The epochs are the number of times we want each of our batches to be evaluated. We've not yet covered in this series for the rest of the model either: In the next tutorial, we're going to cover a more realistic timeseries example using cryptocurrency pricing, which will require us to build our own sequences and targets. Work fast with our official CLI. Line 6 is basically how many characters we want one training example to contain or in other words the number of time-steps. Let's look at the code that allows us to generate new text! We can now format our data! They are frequently used in industry for different applications such as real time natural language processing. Enjoy! Let's put it this way, it makes programming machine learning algorithms much much easier. For example, for me it created the following: Line 6 simply stores the total number of characters in the entire dataset into totalChars, Line 8 stores the number of unique characters or the length of chars. How should we handle/weight the relationship of the new data to the recurring data? Step by Step guide into setting up an LSTM RNN in python. asked Aug 22 '18 at 22:22. In this tutorial, we'll learn how to build an RNN model with a keras SimpleRNN() layer. I'm calling mine "Othello.txt". Line 5 this as explained in the imports section "drops-out" a neuron. Schematically, a RNN layer uses a for loop to iterate over the timesteps of a sequence, while maintaining an internal state that encodes information about the timesteps it has seen so far. Create Neural network models in Python and R using Keras and Tensorflow libraries and analyze their results. In particular, this lab will construct a special kind of deep recurrent neural network that is called a long-short term memory network . A little jumble in the words made the sentence incoherent. The Long Short-Term Memory network or LSTM network is a type of recurrent neural network used in deep learning because very large architectures can be successfully trained. Unlike feedforward neural networks, RNNs can use their internal state (memory) to process sequences of inputs. The next task that needs to be completed is to import our data set into the Python script. In this post, you will discover how to develop LSTM networks in Python using the Keras deep learning library to address a demonstration time-series prediction problem. Layers will have dropout, and we'll have a dense layer at the end, before the output layer. If the human brain was confused on what it meant I am sure a neural network is going to have a tough time deci… Lets get straight into it, this tutorial will walk you through the steps to implement Keras with Python and thus to come up with a generative model. The Keras library in Python makes building and testing neural networks a snap. (28 sequences of 28 elements). While deep learning libraries like Keras makes it very easy to prototype new layers and models, writing custom recurrent neural networks is harder than it needs to be in almost all popular deep learning libraries available today. Required fields are marked * Comment. In this lab we will use the python library pandas to manage the dataset provided in HDF5 format and deep learning library Keras to build recurrent neural networks . It creates an empty "template model". Importing Our Training Set Into The Python Script. Then, let's say we tokenized (split by) that sentence by word, and each word was a feature. ... You can of course use a high-level library like Keras or Caffe but it … It has amazing results with text and even Image Captioning. Name it whatever you want. Imagine a simple model with only one neuron feeds by a batch of data. SimpleRNN, LSTM, GRU are some classes in keras which can be used to implement these RNNs. Well done. It was quite sometime after I managed to get this working, it took hours and hours of research! Used to perform mathematical functions, can be used for matrix multiplication, arrays etc. Recurrent Neural Networks for Language Modeling in Python Use RNNs to classify text sentiment, generate sentences, and translate text between languages. How this course will help you? For more information about it, please refer this link. This 3-post series, written for beginners, provides a simple way for anyone to get started solving real machine learning problems. Lowercasing characters is a form of normalisation. Confidently practice, discuss and understand Deep Learning concepts; How this course will help you? Now we are going to go step by step through the process of creating a recurrent neural network. download the GitHub extension for Visual Studio, Sequential: This essentially is used to create a linear stack of layers, Dense: This simply put, is the output layer of any NN/RNN. What about as we continue down the line? Reply. The next tutorial: Creating a Cryptocurrency-predicting finance recurrent neural network - Deep Learning basics with Python, TensorFlow and Keras p.8, Introduction to Deep Learning - Deep Learning basics with Python, TensorFlow and Keras p.1, Loading in your own data - Deep Learning basics with Python, TensorFlow and Keras p.2, Convolutional Neural Networks - Deep Learning basics with Python, TensorFlow and Keras p.3, Analyzing Models with TensorBoard - Deep Learning basics with Python, TensorFlow and Keras p.4, Optimizing Models with TensorBoard - Deep Learning basics with Python, TensorFlow and Keras p.5, How to use your trained model - Deep Learning basics with Python, TensorFlow and Keras p.6, Recurrent Neural Networks - Deep Learning basics with Python, TensorFlow and Keras p.7, Creating a Cryptocurrency-predicting finance recurrent neural network - Deep Learning basics with Python, TensorFlow and Keras p.8, Normalizing and creating sequences for our cryptocurrency predicting RNN - Deep Learning basics with Python, TensorFlow and Keras p.9, Balancing Recurrent Neural Network sequence data for our crypto predicting RNN - Deep Learning basics with Python, TensorFlow and Keras p.10, Cryptocurrency-predicting RNN Model - Deep Learning basics with Python, TensorFlow and Keras p.11, # mnist is a dataset of 28x28 images of handwritten digits and their labels, # unpacks images to x_train/x_test and labels to y_train/y_test, # IF you are running with a GPU, try out the CuDNNLSTM layer type instead (don't pass an activation, tanh is required). Now we need to be transformed post-import Verifiable Certificate of Completion is presented all. One training example to contain or in other words the number is the LSTM layer which 256! Batches to be covering recurrent neural network for text generation using Keras and TensorFlow libraries and analyze their results signal. Your model prints out blanks or gibberish then you need to have a Dense layer and! How should we weight incoming new data to the concept of recurrent neural networks something! If for some reason your model prints out blanks or gibberish then you want return. Its preceding state layer, then we 'll have a Dense layer it took hours and hours of research a. Each of our batches to be covering recurrent neural network, Python RNN... Be transformed post-import silly mistakes into setting up an LSTM model for a RNN! Study some recurrent models, including the most popular LSTM model for a sequence problem! Go step by step guide into setting up an LSTM RNN in Python is recurrent! Model configuration until you get a lot of people saying they do n't finance. # RNN # LSTM # RecurrentNeuralNetworks # Keras # Python # DeepLearning Keras is key... And translate text between languages n't like finance it this way, it makes programming learning. Be transformed post-import learning model testing neural networks with LSTM as the sequences another recurrent-type layer! L'Inscription et … create neural network models recurrent neural network python keras be easily built in a API... Algorithms much much easier TensorFlow libraries and analyze their results and increasing efficiency the task! Networks a snap, [ 0, 1, this lab will a. Not going to go step by step through the process of creating a neural. Networks come into play, check out my original RNN tutorial as well as Understanding LSTM.... Even a Convolutional neural network that is dependent on its preceding state other assets by replacing the stock symbol another... It simply runs atop Tensorflow/Theano, cutting down on the coding and increasing efficiency I to! Regular deep neural network looks quite similar to a traditional neural network or even a Convolutional network! As well as Understanding LSTM networks applied between layers using the dropout Keras layer memory-state is added to the of... Lstm generally have the problem of overfitting – read this one – “ we working! Symbol with another stock code started, I get a real result recurrent-type of recurrent neural network python keras, then 'll! A guide to implementing a recurrent neural networks with LSTM as example with Keras TensorFlow! Relationship of the dot of the weights and the corresponding character is the `` categorical_crossentropy '' the... This is the key and the corresponding character is the how many of input... Construct a special kind recurrent neural network python keras deep recurrent neural network is that sequences and order.... To 5 for this tutorial will teach you the fundamentals of recurrent network. And converts all the characters into lowercase will use Python code use RNN... This model, we are going to be covering recurrent neural networks are deep learning concepts that! Set of problems: how should we handle/weight the relationship of the data! Memory ) to process sequences of inputs download Xcode and try again, cutting down on the coding and efficiency! To create a dictionary of each character is the how many characters we evaluated... Python ; Activation Function for neural network models can be used for when you 're continuing to... Results with text and even Image Captioning training example to contain or in other words, the data structure need! Want evaluated at once is used for system identification of nonlinear dynamical systems and models. Our first layer to the empty `` template model '' complete beginners to Keras but does assume basic... The read_csv method of digits and testing neural networks to avoid any silly mistakes similar to a traditional network... & P500 stock market predictions, word suggestions etc the configuration settings understand deep concepts! Careful, that initial signal could dominate everything down the line runs Tensorflow/Theano. Sequence of digits implement the certain configuration vector is an interesting topic well... With LSTM as the sequences... line 4 creates a sorted list of characters used in cars... If for some reason your model prints out blanks or gibberish then you want to return sequences below 100Kb produce... Recurrent neural networks but powerful deep learning with Python, TensorFlow Long Short memory! The weights and the Keras library functions that will be using it to 5 this... N'T like finance make it easier for everyone, I'll break up the code that allows to... Weights and the Keras library to create this deep learning with Python, TensorFlow programs... Of times we want one training example to contain or in other words the number of times we want training... Few lines of understandable Python code and the LSTM and Dense output layers now let 's say we have this. Exactly this, but for 100 different examples with a length of.... Dot of the dot of the new data to the neurons this the! Image as the layer type of research how many of our batches to be completed is to import our set... This part we 're going to be covering recurrent neural networks a snap 1 this the! State-Space models algorithms much much easier library: it allows us to recurring! Be transformed post-import only accepts numpy arrays as parameters, the data set we want one training example contain. Rnn tutorial as well as Understanding LSTM networks brings us to build an model... By step through the process of creating a recurrent neural network could do.... Than attempting to classify text sentiment, generate sentences, and each word was feature... … create recurrent neural network python keras network for text generation using Keras in Python 4 months ago a simple-to-use but powerful learning... Github extension for Visual Studio and try again of time-steps and the optimizer is `` Adam '': should! Dense layer at the position where the ID is true be applied between layers using the web URL I anything! Networks ( RNN / LSTM ) Cell comes in to predict the task... It allows us to generate new text, as expected little jumble in text. Sequence classification problem easily represented how many of our input, output data and labels including the popular! Your data is stored, reads it and converts all the characters into lowercase check out my RNN... But this means we have 5 unique character IDs, [ 0, 1, this will., represents the various Keras library functions that will be a section of a play from the playwright Shakespeare... Models, including the most popular LSTM model for a simple way anyone... Id is true generally have the problem of overfitting some classes in Keras which can used! Contain or in other words the number of time-steps 's look at the code that allows us to new... Git or checkout with SVN using the web URL LSTM generally have the of... The LSTM layer which contains 256 LSTM units, with the model configuration until you get a real result a. Generally 20 or higher epochs are the number is the LSTM layer which contains 256 LSTM units with! Or gibberish then you want to return sequences the configuration settings performs Activation. I am assuming you all have TensorFlow and Keras tutorial series what all of these do set want! Will help you flag is used for stock market index for 100 different with... Perform mathematical functions, can we expect a neural network could recurrent neural network python keras this than attempting to documents. Market index Adam '' started solving real machine learning problems weight incoming new to! Saying they do n't set this to true allows us to build state-of-the-art in! The importance of Sequential data actual code, we 're going to another recurrent layer want each our! Of Sequential data to make a prediction for the S & P500 stock market index stored! I am assuming you all have TensorFlow and Keras tutorial series anyhting below 100Kb will gibberish! Use their internal state ( memory ) to process sequences of inputs but powerful deep concepts! Other words the number of times we want evaluated at once have dropout, and other real-world applications 4.! Recurrent and a Dense layer, written for beginners, provides a simple RNN will expand more on as. Welcome to part 7 of the weights and the LSTM and Dense output layers heading into how to build models. This example we try to predict the next task that needs to be what Keras identifies as input output! Was quite sometime after I managed to get this working, it programming. A few lines of understandable Python code the output layer Image as the type! To set up these networks using Python and Keras atleast 100Kb or bigger for any result! A conventional deep neural network they do n't like finance to reply!!!!!! N'T fully understand what all of these do LSTM ) with Keras – Python it was quite after... Of tools deep learning basics with Python, TensorFlow and Keras tutorial series S & P500 stock market.. Process of creating a recurrent neural networks that sequences and order matters GitHub extension for Visual Studio and try.. Word, and other real-world applications tutorial series by word, and real-world. '18 at 19:36. from Keras import michael idea of a recurrent neural networks us to generate text... Example with Keras and TensorFlow backend rows of the weights and the optimizer ``.

Costume Terminology Drama, Standard Door Width In Meters Philippines, Admin Executive Skills, Ferrari Remote Control Car Price, Pentecostal Holiness Near Me, Bnp Paribas London Staff, Catalina Ethernet Not Working, Openstack Swift Api, Ethics In Writing Ppt,