Update README.md

README.md

@@ -8,18 +8,101 @@ tags:
 
 ## Model description
 
-More information needed
+This model is built using two important architectural components proposed by Bryan Lim et al. in  [Temporal Fusion Transformers (TFT) for Interpretable Multi-horizon Time Series Forecasting](https://arxiv.org/abs/1912.09363) called GRN and VSN which are very useful for structured data classification tasks.
 
-## Intended uses & limitations
+1. Gated Residual Networks(GRN) consist of skip connections and gating layers that facilitate information flow efficiently. They have the flexibility to apply non-linear processing only where needed.
+2. Variable Selection Networks(VSN) help in carefully selecting the most important features from the input by getting rid of any unnecessary noisy inputs which could harm the model's performance.
 
-More information needed
+**Note:** This model is not based on the whole TFT model but only uses the GRN and VSN components described in the mentioned paper demonstrating that GRN and VSNs on their own also can be very useful for structured data learning tasks.
+
+## Intended uses 
+
+This model can be used for binary classification task to determine whether a person makes over $500K a year.
 
 ## Training and evaluation data
 
-More information needed
+This model was trained using the [United States Census Income Dataset](https://archive.ics.uci.edu/ml/datasets/Census-Income+%28KDD%29) provided by the UCI Machine Learning Repository. 
+The dataset contains weighted census data extracted from 1994 and 1995 Current Population Surveys conducted by the US Census Bureau.
+The dataset comprises of ~300K samples with 41 input features containing 7 numerical features and 34 categorical features:
+
+| Numerical Features | Categorical Features |
+| :-- | :-- |
+| age | class of worker |
+| wage per hour | industry code |
+| capital gains | occupation code |
+| capital losses | adjusted gross income |
+| dividends from stocks | education |
+| num persons worked for employer | veterans benefits |
+| weeks worked in year | enrolled in edu inst last wk
+|| marital status |
+|| major industry code |
+|| major occupation code |
+|| mace |
+|| hispanic Origin |
+|| sex |
+|| member of a labor union |
+|| reason for unemployment |
+|| full or part time employment stat |
+|| federal income tax liability |
+|| tax filer status |
+|| region of previous residence |
+|| state of previous residence |
+|| detailed household and family stat |
+|| detailed household summary in household |
+|| instance weight |
+|| migration code-change in msa |
+|| migration code-change in reg |
+|| migration code-move within reg |
+|| live in this house 1 year ago |
+|| migration prev res in sunbelt |
+|| family members under 18 |
+|| total person earnings |
+|| country of birth father |
+|| country of birth mother |
+|| country of birth self |
+|| citizenship |
+|| total person income |
+|| own business or self employed |
+|| taxable income amount |
+|| fill inc questionnaire for veteran's admin |
+
 
 ## Training procedure
 
+0. **Prepare Data:** Download the data and convert the target column *income_level* from string to integer and finally split the data into train and validation.
+
+1. **Prepare tf.data.Dataset:** Train and validation datasets created using Step 0 are passed to a function that converts the features and labels into a tf.data.Dataset for training and evaluation.
+
+2. **Define logic for Encoding input features:** All features are encoded while also ensuring that they all have the same dimensionality.
+   
+   - **Categorical Features:** are encoded using *Embedding* layer provided by Keras with output dimension of embedding equal to *encoding_size*
+  
+   - **Numerical Features:** are projected into a *encoding_size* dimensional vector by applying a linear transformation using *Dense* layer provided by Keras
+  
+3. **Implement the Gated Linear Unit (GLU):** consists of two Dense layers where the last last dense layer has a sigmoid activation. GLUs help in suppressing inputs that are not useful for a given task.
+
+4. **Implement the Gated Residual Network:**
+    - Applies Non-linear ELU tranformation on its inputs
+    - Applies linear transformation followed by dropout
+    - Applies GLU and adds the original inputs to the output of the GLU to perform skip (residual) connection
+    - Applies layer normalization and produces the output
+
+5. **Implement the Variable Selection Network:**
+    - Applies a Gated Residual Network (GRN) which was defined in step 4 to each feature individually. 
+    - Applies a GRN for the concatenation of all features followed by a softmax to produce feature weights 
+  - Produces a weighted sum of the output of the individual GRN
+
+6. **Create Model:** 
+- The model will have input layers corresponding to both numerical and categorical features of the given dataset
+- The features received by the input layers are then encoded using the encoding logic defined in Step 2.
+- The encoded features pass through the Variable Selection Network(VSN)
+- The output produced by the VSN are passed through a final *Dense* layer with sigmoid activation to produce the final output of the model
+
+7. **Compile, Train and Evaluate Model**: The model is compiled using Adam optimizer and since the model is meant to binary classification, the loss function chosen is Binary Cross Entropy. 
+The model is trained for 20 epochs and batch_size of 265 with a callback for early stopping. 
+The model performance is evaluated based on the accuracy and loss being observed on the validation set.
+
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training: