code/logs.txt

====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 13:34:36 2022
====================================================================================================
====================================================================================================
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.44822925114896e-06
np.average(doc_lengths) 110.68652000540686
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258


====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 13:38:58 2022
====================================================================================================
====================================================================================================
Dataframe created
Wed Aug  3 13:41:04 2022
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.44822925114896e-06
np.average(doc_lengths) 110.68652000540686
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 13:41:12 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 13:42:59 2022
Wed Aug  3 13:43:19 2022

======== Epoch 1 / 5 ========
Training...


====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 13:46:51 2022
====================================================================================================
====================================================================================================
Dataframe created


====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 13:48:53 2022
====================================================================================================
====================================================================================================
Dataframe created
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 13:49:03 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 13:50:49 2022
Model loaded to GPU
Wed Aug  3 13:51:03 2022

======== Epoch 1 / 5 ========
Training...
  Batch   100  of  26,633. Loss: 0.8224040865898132.   Elapsed: 0:00:20.
0:  becomeYou have to follow the dictates of the Gita. I do not mean that you can bring up his life; but you have to see it from his work. You are a coward. You know his value in his work. But if you take your liberty it will only be a tragedy to befall your child. You are a coward. You see he can carry his work; but his work is done. You would know it was done on your orders to do it, even if it was an act of cruelty. If you carry it yourself, you will never want to be a coward.
  Batch   200  of  26,633. Loss: 0.7663384675979614.   Elapsed: 0:00:43.
0:  Prop the only be of course as your to that is the is I have rendered of your help with the can not to get-hajajar, if was she will be it do will be. You know I not is have been a it will be. Hence the it is of the hope of the hope to a that thatI should of theI.
...
0: XBut the Congress in its recent representation has been not in a hurry to secure its independence. It will remain loyal to the Congress and it will remain a true non-co-operator. It will always be a true non-operator of politics. It can remain in a position of strength. It will be in a position of strength even during the crisis if it is the duty of the Congress to act as an organization. In the same way the Congress has become its own non-operator and an organization may also be its own non-operator of politics. It may remain loyal to the Congress even while it is in a weak state and never fight for independence. But it may be loyal to the Congress at any cost and will, in no way, become non-operator of politics. It has become a part of its own programme, but it will never give up its service to it.
  Batch 2,800  of  26,633. Loss: 0.5805510878562927.   Elapsed: 0:11:35.
0:  stafferThe present position is one of the greatest difficulties I have come across in the course of my travels and the work I do there. It is an important one. The task is difficult but not easy. There is a reason to have been doing the work I have been doing. The work of work is difficult but I have come to the conclusion that it is so difficult that we should do our work only as we would have done it at the present moment. One of my companions in my tour is of course the same opinion. We had a little fun with it a little. It was worth while knowing about it. I had been able to learn some good things by it. The whole thing was done very well. It has gone through many different tests. One of the test is to deal with the effect on a person of one who does a particular thing. That requires patience and a little patience even while doing one thing. One minute we talk about one particular thing, and then think about some


====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 14:05:56 2022
====================================================================================================
====================================================================================================
Dataframe created
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 14:06:06 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 14:07:48 2022
Model loaded to GPU
Wed Aug  3 14:08:05 2022

======== Epoch 1 / 5 ========
Training...
  Batch   100  of  26,633. Loss: 0.8224040865898132.   Elapsed: 0:00:20.


====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 14:09:35 2022
====================================================================================================
====================================================================================================
Dataframe created
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 14:09:43 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 14:11:28 2022
Model loaded to GPU
Wed Aug  3 14:11:46 2022
[['step', '2800'], ['epoch', '0']]
wait

====================================================================================================
====================================================================================================
		=Experiment2= Wed Aug  3 14:13:26 2022
====================================================================================================
====================================================================================================
Dataframe created
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 14:13:36 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 14:15:20 2022
Model loaded to GPU
Wed Aug  3 14:15:34 2022
<All keys matched successfully>
Wed Aug  3 14:15:35 2022

======== Epoch 1 / 5 ========
Training...
  Batch 2,900  of  26,633. Loss: 0.6544240117073059.   Elapsed: 0:00:20.
0:  becomeI had hoped that if I had the time of the moment I could find time for you to learn some foreign proverb, I would have a better way. But the practice of telling your elders as it is so common among our boys and that of asking them to do with it what they may want to do not to do. I have not yet risen in the esteem of my fellow-beings as one does in your case, but I do find that we ourselves are the least fortunate and still in a sense it is the only true sense of service.
  Batch 3,000  of  26,633. Loss: 0.4969857931137085.   Elapsed: 0:00:44.
0:  PropWe have the full picture of the state of things about which you have been told that the people have either done well in a good way or have no knowledge of the real meaning, and so are also the reasons that have arisen for it and they seem like they would have to be called upon to answer them. The problem does include in greater measure the necessity for learning how to handle the question. If a man does know that he knows the meaning of the meaning-of the dictionary, he may enter upon that question without difficulty.
  Batch 3,100  of  26,633. Loss: 0.21492260694503784.   Elapsed: 0:01:09.
0: 47We shall not find anyone who has understood that the Indians need not be angry with us all.


====================================================================================================
====================================================================================================
		=Experiment3= Wed Aug  3 14:19:17 2022
====================================================================================================
====================================================================================================
Dataframe created
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 14:19:24 2022
106,531 training samples
11,837 validation samples
Wed Aug  3 14:21:08 2022
Model loaded to GPU
Wed Aug  3 14:21:22 2022

======== Epoch 1 / 5 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.34002885222435.   Elapsed: 0:03:15.
0:  becomeI have to write for Doctor Sushila Devi. If I write any more about her, and you write about me, I shall probably tell you. But if you get into my letter, it may well go through. How are you?
  Batch 2,000  of  26,633. Loss: 0.8584551811218262.   Elapsed: 0:06:35.

0:  studioI have your letter. The telegram about it does not say anything to me. In view of such matters, I can understand that you had no place at all. But you would certainly have been pleased if I could have sent you a wire. But in the presence of the police, in order to give you notice, it will be good if you could have taken me round you. I certainly wish that you would not be hurt.
  Batch 26,000  of  26,633. Loss: 0.7467376589775085.   Elapsed: 1:26:57.
0: ¯¯This is my views on the question. I do not consider it improper on the part of some of the satyagrahis to hold this view. My reply is not different from yours.

  Average training loss: 0.57
  Training epoch took: 1:29:04

Running Validation...
  Validation Loss: 0.53
  Validation took: 0:03:06

======== Epoch 2 / 5 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.34357091784477234.   Elapsed: 0:03:16.
0: XYou are wrong in using the adjective ‘passive resistance as your name and as your “passive resistance”! It should be well for you to come to me again.
  Batch 2,000  of  26,633. Loss: 0.3667067885398865.   Elapsed: 0:06:36.

0:  AppropriI have your letter. I am glad to see what Kishorelalbhai says. It is not good that your brother has not returned to me. He will be there by some other way if he wishes. He may not even be there at present, but there can be no question about coming here.
  Batch 26,000  of  26,633. Loss: 0.45219725370407104.   Elapsed: 1:27:04.
0:  infectedThe Transvaal British Indian Association has forwarded to the Minister the correspondence that was sent by us at our request.

  Average training loss: 0.51
  Training epoch took: 1:29:11

Running Validation...
  Validation Loss: 0.51
  Validation took: 0:03:06

======== Epoch 3 / 5 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.904615581035614.   Elapsed: 0:03:16.
0:  miserI have your letter. You will certainly get a certificate from the Chief Warder. I see no need for a certificate from a good and distinguished person. Such a certificate is like the latter. A certificate worth a rupee has to be issued from a lawyer. It is necessary for you to produce it from the Sangh. Why cannot you produce it yourself? It is for the Board or the Sangh to produce it. When you produce it without incurring any odium you will be able to produce something cheap and very cheap. You are making yourself at the present moment free of charge. You will not go anywhere without your having a full grasp of the science of khaddar and its working implications. And I shall have no difficulty in making use of the knowledge you have acquired by doing SO. And as soon as you come I shall be in your midst.
  Batch 2,000  of  26,633. Loss: 0.3417126536369324.   Elapsed: 0:06:37.

0: DanDo not take any serious step. It is only after reading the articles on non-co-operation that I feel confident that swaraj is within the power of every one of us. Even if all the co-operation is for all practical purposes and in the shape of swaraj, we can certainly give it. We cannot do without them in spite of the co-operation, that is the case everywhere in India. No doubt, the co-operation is for promoting swaraj. But, if the co-operation is necessary, we shall have to do more work there than here.
  Batch 26,000  of  26,633. Loss: 0.27882975339889526.   Elapsed: 1:26:45.
0:  FirefoxIhave been asked to submit the Congress Scheme. I have not seen it, I have not heard of it. I am sorry for it. It is not a scheme of practical wisdom, it is a scheme of slow and slow torture and wrangling. It is an education Scheme designed to remove all the injustices and humiliations which exist today. It is the scheme of life, the charkha and the charkha that is the test of its power. It is no scheme of purna swaraj. In other words, it is the scheme that has been presented to me by one who has been more than ever convinced of the power of khaddar and khaddar as the means by which the spinning-wheel is the symbol. It is like a man who has been continuously brooding over the use of foreign cloth. I have been listening for two days after the discovery that the charkha has become the only thing that can stop the drain of the ru

  Average training loss: 0.47
  Training epoch took: 1:28:56

Running Validation...
  Validation Loss: 0.50
  Validation took: 0:03:06

======== Epoch 4 / 5 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.3690977394580841.   Elapsed: 0:03:16.
0: icheThe second resolution, the third resolution, has reference only to those who join the satyagraha struggle. I could understand that in view of these three resolutions a special responsibility on their shoulders rests on me. I would like to say that if even five persons attend the Conference I would not find even five persons qualified to offer satyagraha. At the same time I should be satisfied if only five persons participate in the movement. I would like to know if all of them were competent and sufficiently qualified to offer satyagraha. If they do not come, we shall be sorry. In view of the fact that there is a large number of people in the country who agree with me that I am not fit for satyagraha I think that it is the clear duty of the Congress to meet them at a public meeting.
  Batch 2,000  of  26,633. Loss: 0.16660097241401672.   Elapsed: 0:06:37.

0:  aluminumI should know, the same thing, that you do not care in the least about the result. If you take care, nothing may happen to your work. I would not wish, however, that you should take over the responsibility. I do not believe in your thinking that what happened to me was due to you. Whatever things were done at my instance in the past can happen in future. I am not sure if there were no changes.
  Batch 26,000  of  26,633. Loss: 0.18261176347732544.   Elapsed: 1:26:49.
0: elshIt is true that I was angry. But there was no anger behind it. It was not my fault. That being so, I thought I must be able to prove my ahimsa. When anger makes its appearance, it is bound to get a better effect.

  Average training loss: 0.43
  Training epoch took: 1:28:58

Running Validation...
  Validation Loss: 0.50
  Validation took: 0:03:05

======== Epoch 5 / 5 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.37842002511024475.   Elapsed: 0:03:14.
0: �I had said in Bombay that I had no business to publish a book or two on Hindu religion. But, while I was ready to start the movement, I was pressed for time. There was an agitation in which all religious leaders were present and were asked to give their opinions freely. One of them expressed his inability to give such a request, and the leaders were not allowed to give their views freely. A few days later I was taken to task for not giving the views that I have been able to form on Hindu religion.
  Batch 2,000  of  26,633. Loss: 0.24595755338668823.   Elapsed: 0:06:33.

0: izerThe Indian community will never forget their struggle, which was not merely to secure the repeal of the law but to make it a point to oppose the Asiatic Act. To put it more conversely, if we have any hope of winning, it will not be by going to gaol. There will be, on the whole, some saving for having to face hardships, but the struggle will remain the same.
  Batch 26,000  of  26,633. Loss: 0.3665473461151123.   Elapsed: 1:26:27.
0: FullGANDHI: It does not surprise me, nor is it surprising that you should find it difficult to answer the question of Hindu-Muslim unity.

  Average training loss: 0.39
  Training epoch took: 1:28:34

Running Validation...
  Validation Loss: 0.50
  Validation took: 0:03:05

Training complete!
Total training took 7:40:12 (h:mm:ss)
Wed Aug  3 22:01:34 2022
       Training Loss  Valid. Loss Training Time Validation Time
epoch                                                          
1               0.57         0.53       1:29:04         0:03:06
2               0.51         0.51       1:29:11         0:03:06
3               0.47         0.50       1:28:56         0:03:06
4               0.43         0.50       1:28:58         0:03:05
5               0.39         0.50       1:28:34         0:03:05
The GPT-2 model has 148 different named parameters.

==== Embedding Layer ====

transformer.wte.weight                                  (50259, 768)
transformer.wpe.weight                                   (1024, 768)

==== First Transformer ====

transformer.h.0.ln_1.weight                                   (768,)
transformer.h.0.ln_1.bias                                     (768,)
transformer.h.0.attn.c_attn.weight                       (768, 2304)
transformer.h.0.attn.c_attn.bias                             (2304,)
transformer.h.0.attn.c_proj.weight                        (768, 768)
transformer.h.0.attn.c_proj.bias                              (768,)
transformer.h.0.ln_2.weight                                   (768,)
transformer.h.0.ln_2.bias                                     (768,)
transformer.h.0.mlp.c_fc.weight                          (768, 3072)
transformer.h.0.mlp.c_fc.bias                                (3072,)
transformer.h.0.mlp.c_proj.weight                        (3072, 768)
transformer.h.0.mlp.c_proj.bias                               (768,)

==== Output Layer ====

transformer.ln_f.weight                                       (768,)
transformer.ln_f.bias                                         (768,)
Saving model to /media/data_dump/Ritwik/ggpt/model_save/

Wed Aug  3 23:14:22 2022
tensor([[50257,   314,  4601,   284,   910,   326]], device='cuda:0')
0:  I wish to say that at the present time this campaign is a test for us all. If you accept my argument that, while a community must not only not suffer in order to save itself from slavery but for its own good, the whole community must suffer morally, through the struggle of satyagraha, then only can we win our freedom. We should also know that in this struggle the very opposite position is implied by non-violence. Our struggle consists in non-violence of the strong. When a thing loses its savour, we must realize that it is not because we did not know how to do it or that we did not have strength enough to defend it; we should therefore realize that we cannot succeed even if we follow the path of violence. Our struggle depends on the strength which we possess. In such strength alone lies our victory. The struggle of the Transvaal Indians requires no weapons other than truth and non-violence.


1:  I wish to say that you and Maganlal should keep perfect quiet now. Do not at all worry on your account. In a way Maganlal will have to look after everything. As long as he can do it he will continue to do as he wishes. If you two are firm in your vow, you should never worry and also not get angry with one another. If Maganlal willingly allows a few of us to be angry with him we should feel happy. For the sake of that love, we should put out each other. This does not, however, mean that I should not feel unhappy about the affair on your account. On the other hand, if we wish to enjoy the happy in Phoenix we should look upon it as an opportunity for peace. If in future we have to worry about something we cannot remove it we shall certainly get away from it. We cannot escape it if we are indifferent about the Phoenix way or if our interests are involved in the Phoenix way or have to hurt either of them. Anybody who observes the Phoenix way must find some peace there too. But to go to Phoenix means to go there to serve the Phoenix life. I would prefer that you should live with Maganlal and be in Phoenix for some time now. If, however, you would not like to stay there for two years and you think that we would not be better if you are sent to Phoenix, I would prefer that you should spend two years there. For those who want to live permanently in Phoenix you can only have to keep in reserve one room at Phoenix. For Maganlal we can provide one room at Phoenix. We want to make the best use of it. If Maganlal has to be in Phoenix for long, he must live on the land. And so, we can make arrangements to meet every need. We should also make arrangements for keeping everything clean and keep some cleanliness in our lives. If Maganlal intends to have Manilal here, let him come and live here. Let him not think that his health has been undermined. We should not take too much notice of him even if he is a source of worry.


2:  I wish to say that both of you have to take care of your health. The letter about Jayaprakash has been given to me only by Jayaprakash. I am not going to find him with me in the jail. Do what needsful.


Wed Aug  3 23:14:30 2022




====================================================================================================
====================================================================================================
		=Experiment4= Wed Aug  3 23:33:47 2022
====================================================================================================
====================================================================================================
all_tc_sents_500.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.30923654734603e-06
np.average(doc_lengths) 108.86547346029847
Wed Aug  3 23:35:27 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Wed Aug  3 23:35:40 2022
108,313 training samples
12,035 validation samples
Wed Aug  3 23:37:07 2022
Model loaded to GPU
Wed Aug  3 23:37:19 2022


====================================================================================================
====================================================================================================
		=Experiment4= Thu Aug  4 10:23:20 2022
====================================================================================================
====================================================================================================
all_tc_sents_500.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.30923654734603e-06
np.average(doc_lengths) 108.86547346029847
Thu Aug  4 10:25:11 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Thu Aug  4 10:25:21 2022
108,313 training samples
12,035 validation samples
Thu Aug  4 10:26:58 2022
Model loaded to GPU
Thu Aug  4 10:27:10 2022

======== Epoch 1 / 5 ========
Training...
  Batch 1,000  of  27,079. Loss: 0.670301079750061.   Elapsed: 0:03:15.
0:  becomeI have given you my full confidence in your ability to respond to your letters. I hope you are not overmuch frightened by a telegram from Calcutta. Just now you are trying hard enough to reach the Gosev temple. However, if your fever has subsided, I hope you will not have to face it in future. The Government is all right. But if you can get overmuch I shall not hesitate to give you the best guidance I can. It is a great cause for your fear. There is no harm in saying anything which will have a lasting effect on the people or on you. But let me tell you a true faith has a definite place in it. I have known a number of Mussulmans who have believed in God. They believe in Him only through Him. If we have a true faith in Him, He will be the foundation of every true religion. Every Hindu, Hindu and Muslim is equal. The only one who cannot believe in God is
  Batch 2,000  of  27,079. Loss: 0.5499960780143738.   Elapsed: 0:06:34.

0: GunEven in that case the Government have given rise to such a row. A general rule cannot be accepted. Even the administration of Defence can be decided from an oversight point of view. It is our duty to take measures on this, i.e., we should go on acting in the manner we might wish. We should be in the same position as the Government in that we are in the country. If our country were really a country of love, I should certainly have said much. We should be satisfied with a rule which cannot but be an act of love. I would say that I am prepared to make such a statement. What I have said on this matter is of the greatest benefit to us.
  Batch 25,000  of  27,079. Loss: 0.6164011359214783.   Elapsed: 1:36:21.
0:  studioIn my opinion, there is no place for a single British Indian to be so far as I know. The British Government are not bound to allow any Indian to come to India, and that there is no harm in any such rule being made. If British Indians hold on under the existing law they must refuse to submit to it. If it is held of a large number, they should continue to do so. I dont know that any Indian should go to Bombay. I dont know that many Indians have been doing so. And it would be wrong on my part if a British Indian had to go to Bombay in the course of a British rule. But the matter is just now on a purely humanitarian question. There is nothing to be called such a case. And the Government has not been able to give me courage to fight for this injustice. For example, my son was a Muslim; I was in the same position. I told him that I had a son, his father, but that he


====================================================================================================
====================================================================================================
		=Experiment4= Thu Aug  4 15:12:45 2022
====================================================================================================
====================================================================================================
all_tc_sents_500.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.30923654734603e-06
np.average(doc_lengths) 108.86547346029847
Thu Aug  4 15:15:11 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Thu Aug  4 15:15:11 2022
108,313 training samples
12,035 validation samples
Thu Aug  4 15:17:06 2022
Model loaded to GPU
Thu Aug  4 15:17:19 2022
<All keys matched successfully>
Thu Aug  4 15:17:20 2022

======== Epoch 1 / 5 ========
Training...
  Batch 26,000  of  27,079. Loss: 0.473719984292984.   Elapsed: 0:03:24.
0:  becomeThe resolution was passed by the Legislative Council at Belgaum. There is, then, no doubt that the Hindu-Muslim question is still a small affair. There can be no misunderstanding about the case of the Vykom, whether it includes in the Vaishnava community a large number of Antyajas, or any other community as the case may be. The public are not aware of this very movement, because, in many cases, there never was much difference between the two communities. I have seen many Hindus in the South feel that they are the only thing that they can do to remove this sin of untouchability.
  Batch 27,000  of  27,079. Loss: 0.3604016900062561.   Elapsed: 0:06:52.
0:  PropI am not going to worry. The time-table is that you should not make any provision about it and if you wish to make a promise, you should make it.

  Average training loss: 0.04
  Training epoch took: 0:07:11

Running Validation...
  Validation Loss: 0.54
  Validation took: 0:03:17

======== Epoch 2 / 5 ========
Training...
  Batch 1,000  of  27,079. Loss: 0.621677577495575.   Elapsed: 0:03:22.
0: 47I shall certainly seek your help and give you full satisfaction if all the money is saved. I wish I could write the following reply :
  Batch 2,000  of  27,079. Loss: 0.36880502104759216.   Elapsed: 0:06:47.

0:  stafferI do not know the facts of the case. If Mr. Ritch were a true Christian he would have set out with a fair name as a child. If he had to leave South Africa I am sure he would have.
  Batch 27,000  of  27,079. Loss: 0.3913257122039795.   Elapsed: 1:33:08.
0:  recipesI have your two letters. I did think that you were not coming, but I knew that you would come, and so I thought that if it was not well meant, I should certainly like to see you. But I had no intention of staying there. So we had a discussion with Kishorelal. However, I did not feel inclined to discuss the matter here. Your decision about leaving does not appeal to me.

  Average training loss: 0.51
  Training epoch took: 1:33:29

Running Validation...
  Validation Loss: 0.51
  Validation took: 0:03:17

======== Epoch 3 / 5 ========
Training...
  Batch 1,000  of  27,079. Loss: 0.19190533459186554.   Elapsed: 0:03:22.
0:  admissionsLet us see what is happening. You have to put your hands to the wheel. If you are not going to learn it through the teacher, you may start learning it by heart.
  Batch 2,000  of  27,079. Loss: 0.2705977261066437.   Elapsed: 0:06:46.
0: reetOne thing that has been uppermost in my mind is this. Though my fast was of a religious nature it may be that it is a religious purpose, for me my fast was political, or it may not be religious. I have written so much in the past about myself that I cannot say that there is any spiritual end in this fast. I cannot do anything which would mean that the motive behind the fast was not selfish but purely religious. I would have been able to go into the details and found out the truth of that decision. Such was my position. There were no religious convictions in my mind but there was no religious convictions about me. If one sees this truth, there is no other end in view. I would have been wrong in my own self. But I do not regard the fast as political. I cannot be an untruthful man or a true man. The idea behind the fast was not to harm anybody.
  Batch 3,000  of  27,079. Loss: 0.277152955532074.   Elapsed: 0:10:15.

0:  relationYou should have no hesitation in giving up everything. You should give up whatever you have undertaken. If you have an opportunity, you should forget the desire to come out. I must admit that if my life falls short of you, I would welcome a visit. But even if I do not get any help, you will have to give up all fear. What can I say to you? I know your love. It is enough that you are with me.
  Batch 27,000  of  27,079. Loss: 0.3280184864997864.   Elapsed: 1:32:55.
0:  LisaOne way you can help for Harijan, namely, to be patient and correct in your conduct. I see that you will have to suffer for the cause of Harijan.

  Average training loss: 0.47
  Training epoch took: 1:33:14

Running Validation...
  Validation Loss: 0.50
  Validation took: 0:03:17

======== Epoch 4 / 5 ========
Training...
  Batch 1,000  of  27,079. Loss: 0.7236179113388062.   Elapsed: 0:03:22.
0: darLet us see what can be done for the people. I have already told the Hindus in my speeches that they have not got their homes in their own homes. I have even said that they should live in their locality. The Muslim friends there are going to take them away, and they are giving their foodstuffs to the Hindus. If we want to keep their homes clean it is our duty to provide them with houses to live in. Why should they be made naked when they leave our houses? That is what I have explained here. If they want to ask their neighbours for houses they have to be satisfied with only one room at a time. It is not enough that they provide houses to the Hindus. It is necessary that the houses should not be dirty.
  Batch 2,000  of  27,079. Loss: 0.34862542152404785.   Elapsed: 0:06:50.

0:  vomI am not prepared to admit even the existence of these young men or my co-workers. I believe that non-co-operation as well as spinning and weaving are part of varnashramadharma. I do not believe that these four conditions need to be accepted as essential.
  Batch 27,000  of  27,079. Loss: 0.2437172532081604.   Elapsed: 1:33:06.
0:  PhilosophyI had your letter. The second letter seems to have crossed mine. I do not wish to do any work, nor must I give up what little I have gained. The main reason is my weakness, and therefore lack of faith in the work that I have undertaken. If you want that I should go and see you I shall not be able to do it and, in any case, my health cannot be looked after.

  Average training loss: 0.44
  Training epoch took: 1:33:26

Running Validation...
  Validation Loss: 0.49
  Validation took: 0:03:17

======== Epoch 5 / 5 ========
Training...
  Batch 1,000  of  27,079. Loss: 0.24802394211292267.   Elapsed: 0:03:22.
0:  undertakenThe question raised in this letter is quite easy. If my writings are not written in anger or with an uncharitable intent, I shall be betrayed into an ignorant or ignorant hasty hasty or ignorant construction of them. I have not written with the intention of correcting a single error in my writings. When I first wrote a long letter to Shri Jagdish Chandra Bose on this subject, he had suggested that I should write an article for it. I had not prepared the article. Now I am writing a short one after correcting a single bad page. However, since this is written with my help, I shall send the thing to you before you return it.
  Batch 2,000  of  27,079. Loss: 0.23161667585372925.   Elapsed: 0:06:49.

0: ."[I have read your letter to Jamnalalji. We are living in an entirely new place. I am glad that you are doing the work which you are doing. Do write regularly. Have you started learning carding and spinning and are you studying any day?
  Batch 27,000  of  27,079. Loss: 0.18835848569869995.   Elapsed: 1:33:19.
0:  NazHere I have only one question, i. e., about the propriety or otherwise of the Constituent Assembly. This one should have no connection with the question of the Princes.

  Average training loss: 0.40
  Training epoch took: 1:33:38

Running Validation...
  Validation Loss: 0.50
  Validation took: 0:03:18

Training complete!
Total training took 6:37:25 (h:mm:ss)
Thu Aug  4 21:54:44 2022
       Training Loss  Valid. Loss Training Time Validation Time
epoch                                                          
1               0.04         0.54       0:07:11         0:03:17
2               0.51         0.51       1:33:29         0:03:17
3               0.47         0.50       1:33:14         0:03:17
4               0.44         0.49       1:33:26         0:03:17
5               0.40         0.50       1:33:38         0:03:18
The GPT-2 model has 148 different named parameters.

==== Embedding Layer ====

transformer.wte.weight                                  (50259, 768)
transformer.wpe.weight                                   (1024, 768)

==== First Transformer ====

transformer.h.0.ln_1.weight                                   (768,)
transformer.h.0.ln_1.bias                                     (768,)
transformer.h.0.attn.c_attn.weight                       (768, 2304)
transformer.h.0.attn.c_attn.bias                             (2304,)
transformer.h.0.attn.c_proj.weight                        (768, 768)
transformer.h.0.attn.c_proj.bias                              (768,)
transformer.h.0.ln_2.weight                                   (768,)
transformer.h.0.ln_2.bias                                     (768,)
transformer.h.0.mlp.c_fc.weight                          (768, 3072)
transformer.h.0.mlp.c_fc.bias                                (3072,)
transformer.h.0.mlp.c_proj.weight                        (3072, 768)
transformer.h.0.mlp.c_proj.bias                               (768,)

==== Output Layer ====

transformer.ln_f.weight                                       (768,)
transformer.ln_f.bias                                         (768,)
Saving model to /media/data_dump/Ritwik/ggpt/model_save/
Model and tokenizer loaded!
Thu Aug  4 21:54:46 2022
tensor([[50257,   314,  4601,   284,   910,   326]], device='cuda:0')
0:  I wish to say that, for the first two reasons I cannot make a complete list of the places in Kathiawar.


1:  I wish to say that it would be a very good thing if you can serve the people while maintaining contact with the villagers. The villagers who are carrying on this work should be the most fortunate. We should give up our infatuation, if there be any, with the city people and serve them. There is nothing wrong in serving them. There is no harm if we do it. I have no doubt at all that they are the best workers. If there was none, the people would never have been able to do the work they did. We have not acquired any knowledge about the subject of village sanitation. In spite of our infatuation with the city people, we do not have a model hospital for the poor. In our own cities too, we do not give proper attention to rural sanitation. Wherever we go, we do not see anything wrong in it. Where there are people who are the aggressors and, being frightened by them, drive them away, we should think and plead with them to desist. It would be better if we set our heart on doing this work, and start with the effort. People cannot develop a true desire for knowledge in their villages. It is only when we have such an atmosphere in our hearts that we can cultivate the desire for service. Such service requires money. If all the villagers could improve their ways of living, the work would be easy. This is what I tell all of you. Do what is absolutely necessary. If you wish to give a hundred rupees for the building, do it yourself, but do not ask for more.


2:  I wish to say that even now I do not know the real meaning of satyagraha, which is, to say that violence in self-defence is a weapon and a weapon of the weak and the helpless. In South Africa, unarmed men were given the use of arms. In India there are numerous examples in which arms and the sword used against a supposed enemy of reformers rendered them useless. In the other parts of Europe, such use of arms is resorted to with great courage. In Europe, though India possesses arms, there are still a few things which are not ordinarily possessed. In this way there is a great gulf between the two communities. I do not know whether in South Africa, Kheda and Champaran, there is any such difference as in Russia. In Champaran and Kheda, the arms used against the Khedas are not of the highest order. At Kheda, it is only men who can exercise the sword. The Konkani Pathans and the Konkanis fought with one another but they were defeated. In Champaran, the Konkanis and the Hindus fought with the Kheda and the Muslims fought with the Kheda. How is this to be described as satyagraha? A Kheda is a very brave people. He never crumbles in fighting. He does not, like a mouse, fight because he knows how to fight. One who is afraid of going to kill may be afraid of dying. What kind of fighting is it? One who has learnt to fight must be brave.


Thu Aug  4 21:54:52 2022




====================================================================================================
====================================================================================================
		=Experiment5= Fri Aug  5 02:16:42 2022
====================================================================================================
====================================================================================================
all_tc_sents_200.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 6.715195143570872e-06
np.average(doc_lengths) 87.98077439630396
Fri Aug  5 02:18:55 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Fri Aug  5 02:18:55 2022
134,024 training samples
14,892 validation samples
Fri Aug  5 02:20:52 2022
Model loaded to GPU
Fri Aug  5 02:21:06 2022

======== Epoch 1 / 5 ========
Training...
  Batch 1,000  of  33,506. Loss: 0.5928560495376587.   Elapsed: 0:03:22.
0:  becomeIt was a fine evening. The two boys had to sit in the middle at the table and watch the news and discuss it with the girls, and the mother who lived in the adjoining house also came to see them. It was, moreover, an ordinary evening, for a girl in a white robe to do a job. Her husband did not look at her, so she did not go to the room, but went to the adjoining room and sat in one, and so on. The two boys sat in the middle, and so on. When the evening was finished, they went to the doctor. He was not in there, but the doctor told them to write to their doctor.
  Batch 2,000  of  33,506. Loss: 0.3896593749523163.   Elapsed: 0:06:48.

0:  shortagesI know that some very fine boys have come. The reason for this is that I never get visitors, nor have I a moments leisure. Today I am doing a great thing for the sake of God.
  Batch 33,000  of  33,506. Loss: 0.3922334909439087.   Elapsed: 1:53:54.
0:  depictAnd if the workers, whether in the shape of their own houses or in their own homes, do not work for themselves, they must be entirely free. If they had any other help in making them self-reliant, they would have nothing but their own freedom. It is more than their rights, their maintenance and the maintenance of their property. But their duties, for this one can only be their own and their own.

  Average training loss: 0.45
  Training epoch took: 1:55:40
Fri Aug  5 04:16:47 2022

Running Validation...
  Validation Loss: 0.43
  Validation took: 0:04:04

======== Epoch 2 / 5 ========
Training...
  Batch 1,000  of  33,506. Loss: 0.3831220865249634.   Elapsed: 0:03:22.
0: ettingsYou should not even think of marrying off her. The best way is to marry off your brother and a sister. You would certainly serve your father by serving him. You will never marry anybody else. Do not take care of your own health.
  Batch 2,000  of  33,506. Loss: 0.40029051899909973.   Elapsed: 0:06:49.

0:  ControlI have now got some quiet time. A patient can certainly perform a great operation but he ought not to do it unless he himself is in earnest. It is difficult for a man to get it, but it is the utmost he should do before doing it. A patient should be a doctor. The patient should feel easy and healthy, and should have a good look in his life. He has enough strength to be healthy, and he takes care of his health. But, for the present, the thing is to see him to the point of death, so that he may get well.
  Batch 33,000  of  33,506. Loss: 0.3986192047595978.   Elapsed: 1:53:54.
0: 246Do come if you want. I shall feel most comfortable in any hotel where I am not averse to my staying a week.

  Average training loss: 0.41
  Training epoch took: 1:55:39
Fri Aug  5 06:16:30 2022

Running Validation...
  Validation Loss: 0.41
  Validation took: 0:04:04

======== Epoch 3 / 5 ========
Training...
  Batch 1,000  of  33,506. Loss: 0.37087905406951904.   Elapsed: 0:03:22.
0: ){I could not find time to talk much about the difficulties arising from the settlement. As I did not know that they had involved too much trouble to speak, I left for Delhi where I took my second morning train at night and met the people at the station. In this place, too, I found time to write a great deal.
  Batch 2,000  of  33,506. Loss: 0.5901212096214294.   Elapsed: 0:06:51.

0:  DefenseIf, after knowing the truth, a person who is innocent of sin commits no sin, who can save him? But no one can save an innocent person who has been ruined in the name of religion, for in order to save a single individual the best of sinners has to become good and good. If one man has fallen into the vice, he is in the position of a lecherous creature. In a country where people are wicked, there is no dharma at all for a man who is wicked. Hence it is that we cannot protect ourselves from ourselves. For one thing we cannot protect society if we have not become good. For another thing we have to become wicked.
  Batch 33,000  of  33,506. Loss: 0.2512035369873047.   Elapsed: 1:54:00.
0: ysseyThis is a question not of the propriety of the Act and not of satyagraha or any other kind of civil resistance that I could think of. If you have any difficulty, you will get the answer by yourself and not me. But if you see the difficulty of my argument, please ask me.”

  Average training loss: 0.37
  Training epoch took: 1:55:47
Fri Aug  5 08:16:21 2022

Running Validation...
  Validation Loss: 0.40
  Validation took: 0:04:05

======== Epoch 4 / 5 ========
Training...
  Batch 1,000  of  33,506. Loss: 0.29394030570983887.   Elapsed: 0:03:22.
0:  turkeyWe should have pity on our brethren. Our duty is to try to understand the situation. We should try to remove those who have come back to work, to persuade them to return, to appeal to the Government and win their hearts by carrying on the work even if they should lose their lives, and to persuade them that they are committing a sin. A coward cannot always appeal to people. One who holds his life in his hands can only act in a way which appeals to him.
  Batch 2,000  of  33,506. Loss: 0.3915692865848541.   Elapsed: 0:06:49.
0: ceedYou should study the enclosed circular carefully. I am writing this under great pressure of work and have not been able to send for more hours at my disposal.
  Batch 3,000  of  33,506. Loss: 0.36223453283309937.   Elapsed: 0:10:16.

  Batch 33,000  of  33,506. Loss: 0.41078129410743713.   Elapsed: 1:53:55.
0:  hedIn order to remove this blot on Hinduism, Hindus and Mussalmans should consider it a sin to be touch-able and touch a woman. This is no small sin. I have been known throughout my whole life and have been connected with all the Hindus. I claim to be among the very few Hindus who would touch me as a child. But I feel that this is a farce and an unwarranted assumption. I have been accused of a great deal of untruthfulness, of exaggeration and of cowardice. I must admit that I admit that this is being done. But I have still not been guilty of it.

  Average training loss: 0.34
  Training epoch took: 1:55:45
Fri Aug  5 10:16:10 2022

Running Validation...
  Validation Loss: 0.40
  Validation took: 0:04:04

======== Epoch 5 / 5 ========
Training...
  Batch 1,000  of  33,506. Loss: 0.217268705368042.   Elapsed: 0:03:22.
0:  smarterThere is no lack of understanding in my article on this subject. This friend has raised two doubts about my meaning.
  Batch 2,000  of  33,506. Loss: 0.27171939611434937.   Elapsed: 0:06:47.
0: smartIt seems to me, however, that if the Indian community in South Africa wishes to hold the Conference, the Conference at Pretoria is at the very threshold. The Indian community is extremely disinclined to hold it, and if possible, to hold the Conference at Johannesburg. All this, I consider to be a matter of time. It may be necessary for me to go further. The conference may not be held in Pretoria for the purpose of attending the Conference; but my only desire is to hold the Conference at Johannesburg and, if possible, go there at the end of the week. However, if you have any doubt, please telephone at Pretoria.
  Batch 3,000  of  33,506. Loss: 0.19902575016021729.   Elapsed: 0:10:15.
...
...
0: PostedBut the spinning-wheel has come in for much ridicule. You might think that it is the only thing worth seeing which would satisfy me. Even then I see the thing quite differently. It is the only thing worth seeing for the country. The world is watching and also preparing us for it. We should all, men, women and children, spin regularly. For the whole of India, it is possible to hold a fairly good number of people together during this year of grace.
  Batch 33,000  of  33,506. Loss: 0.2683935761451721.   Elapsed: 1:53:52.
0:  bladesI am sorry for you and the other members of the Working Committee but I know that you do not consider it right. I would like you to realize my difficulty in reaching the conclusion you have arrived at. If you have come to any conclusion you have no business to hesitate to tell me so. I would like you to appreciate the difficulties that I have been obliged to place before you in connection with your decision to take an immediate step.

  Average training loss: 0.31
  Training epoch took: 1:55:41
Fri Aug  5 12:15:55 2022

Running Validation...
  Validation Loss: 0.41
  Validation took: 0:04:05

Training complete!
Total training took 9:58:53 (h:mm:ss)
Fri Aug  5 12:19:59 2022
       Training Loss  Valid. Loss Training Time Validation Time
epoch                                                          
1               0.45         0.43       1:55:40         0:04:04
2               0.41         0.41       1:55:39         0:04:04
3               0.37         0.40       1:55:47         0:04:05
4               0.34         0.40       1:55:45         0:04:04
5               0.31         0.41       1:55:41         0:04:05
The GPT-2 model has 148 different named parameters.

==== Embedding Layer ====

transformer.wte.weight                                  (50259, 768)
transformer.wpe.weight                                   (1024, 768)

==== First Transformer ====

transformer.h.0.ln_1.weight                                   (768,)
transformer.h.0.ln_1.bias                                     (768,)
transformer.h.0.attn.c_attn.weight                       (768, 2304)
transformer.h.0.attn.c_attn.bias                             (2304,)
transformer.h.0.attn.c_proj.weight                        (768, 768)
transformer.h.0.attn.c_proj.bias                              (768,)
transformer.h.0.ln_2.weight                                   (768,)
transformer.h.0.ln_2.bias                                     (768,)
transformer.h.0.mlp.c_fc.weight                          (768, 3072)
transformer.h.0.mlp.c_fc.bias                                (3072,)
transformer.h.0.mlp.c_proj.weight                        (3072, 768)
transformer.h.0.mlp.c_proj.bias                               (768,)

==== Output Layer ====

transformer.ln_f.weight                                       (768,)
transformer.ln_f.bias                                         (768,)
Saving model to /media/data_dump/Ritwik/ggpt/model_save/
Model and tokenizer loaded!
Fri Aug  5 12:20:01 2022
tensor([[50257,   314,  4601,   284,   910,   326]], device='cuda:0')
0:  I wish to say that the workers do not understand this simple thing. If what they are saying is true, it is sad. I have never worked in this manner; I have not had the opportunity to go into this matter. But I also wish that the workers would learn some lesson in the Ashram and get experience. In other words, the Ashram is not meant for earning money. It is meant for developing the intellect.


1:  I wish to say that no one should think what is happening here because the Government has committed a mistake. It is a mistake to say that we shall not be able to manage the affairs of the whole country. We shall only be responsible for them. No one can be blamed for what happened at Allahabad or other places. We can have no reason to be unhappy in this manner. No one is fit enough to go anywhere with a permit. We shall only have to assume that there is some risk to take.


2:  I wish to say that it is a matter of sorrow and shame for us that the Hindus have not been able to do justice to the cow. We are faced with starvation. The Hindu Shastras do not permit the slaughter of cows for religious purposes. Even the sacred name of the Shastras has no place in our religious code. It cannot have the sanctity of a temple. The scriptures of the world tell us that all who visited the temple at one time were not Hindus. They say that, even if there were no temple in this age, one could visit it. We must not imitate them. It is impossible that a majority of people in India can be given the purification they require, because of what they say.


Fri Aug  5 12:20:04 2022


====================================================================================================
====================================================================================================
		=Experiment6= Fri Aug  5 16:28:59 2022
====================================================================================================
====================================================================================================
/media/nas_mount/Ritwik/Ai4Bharat_text_corpora/data/en/en_clean.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 1.382733252100489e-06
np.average(doc_lengths) 25.024068862740805
Fri Aug  5 19:05:15 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Fri Aug  5 19:05:15 2022
38,402,200 training samples
4,266,912 validation samples
Sat Aug  6 00:11:29 2022
Model loaded to GPU
Sat Aug  6 00:12:17 2022

======== Epoch 1 / 3 ========
Training...
  Batch 1,000  of  4,800,275. Loss: 0.2311265915632248.   Elapsed: 0:04:40.
0:  become“He said that his house is in the village for the entire region," the district administration said.
  Batch 2,000  of  4,800,275. Loss: 0.33976125717163086.   Elapsed: 0:09:01.
0:  Prop“He says “I do not even want this happening in the second day.
  Batch 3,000  of  4,800,275. Loss: 0.2708880603313446.   Elapsed: 0:13:23.
  ....
  ....
  Batch 2,996,000  of  4,800,275. Loss: 0.24251170456409454.   Elapsed: 8 days, 3:53:55.
0:  BulThis can help you score runs when you are not working or when you start batting at 5.
  Batch 2,997,000  of  4,800,275. Loss: 0.24261796474456787.   Elapsed: 8 days, 3:57:57.
0:  GoodIn December, when the Supreme Court asked the Centre to allow it to withdraw from the GST, it told the government that the Centre would not be able to make the new rates unchanged.


====================================================================================================
====================================================================================================
		=Experiment6= Fri Aug 26 19:11:26 2022
====================================================================================================
====================================================================================================
/media/nas_mount/Ritwik/Ai4Bharat_text_corpora/data/en/en_clean.txt
Dataframe created
Fri Aug 26 19:13:26 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Fri Aug 26 19:13:26 2022
38,402,200 training samples
4,266,912 validation samples
Fri Aug 26 19:13:38 2022
Model loaded to GPU
Fri Aug 26 19:13:57 2022
<All keys matched successfully>
Fri Aug 26 19:14:01 2022

======== Epoch 1 / 1 ========
Training...
  Batch 2,998,000  of  4,800,275. Loss: 0.15963533520698547.   Elapsed: 2:01:04.
0:  becomeThis had happened when he was the chief justice of Pakistan.
  Batch 2,999,000  of  4,800,275. Loss: 0.25272271037101746.   Elapsed: 2:04:59.
0:  PropThe new study comes months after the second largest cancer in the history of the global field, and is now being used as a test to determine how cancer cells form in cancer cells.
  Batch 3,000,000  of  4,800,275. Loss: 0.2565567195415497.   Elapsed: 2:08:54.
...
...
...
0: ,-In the letter, Kejriwal sought action against the AAP dispensation in Delhi.
  Batch 4,800,000  of  4,800,275. Loss: 0.1904856413602829.   Elapsed: 5 days, 1:45:51.
0:  DeclI will go to the people from every corner of the country.

  Average training loss: 0.08
  Training epoch took: 5 days, 1:47:01
Wed Aug 31 21:01:03 2022

Running Validation...
  Validation Loss: 0.21
  Validation took: 11:35:47

Training complete!
Total training took 5 days, 13:22:52 (h:mm:ss)
Thu Sep  1 08:36:49 2022
       Training Loss  Valid. Loss    Training Time Validation Time
epoch                                                             
1               0.08         0.21  5 days, 1:47:01        11:35:47
The GPT-2 model has 148 different named parameters.

==== Embedding Layer ====

transformer.wte.weight                                  (50259, 768)
transformer.wpe.weight                                   (1024, 768)

==== First Transformer ====

transformer.h.0.ln_1.weight                                   (768,)
transformer.h.0.ln_1.bias                                     (768,)
transformer.h.0.attn.c_attn.weight                       (768, 2304)
transformer.h.0.attn.c_attn.bias                             (2304,)
transformer.h.0.attn.c_proj.weight                        (768, 768)
transformer.h.0.attn.c_proj.bias                              (768,)
transformer.h.0.ln_2.weight                                   (768,)
transformer.h.0.ln_2.bias                                     (768,)
transformer.h.0.mlp.c_fc.weight                          (768, 3072)
transformer.h.0.mlp.c_fc.bias                                (3072,)
transformer.h.0.mlp.c_proj.weight                        (3072, 768)
transformer.h.0.mlp.c_proj.bias                               (768,)

==== Output Layer ====

transformer.ln_f.weight                                       (768,)
transformer.ln_f.bias                                         (768,)
Saving model to /media/data_dump/Ritwik/ggpt/model_save/
all_tc_sents_768.txt
Dataframe created
len(doc_lengths[doc_lengths > 768])/len(doc_lengths) 8.44822925114896e-06
np.average(doc_lengths) 110.68652000540686
Thu Sep  1 08:38:35 2022
The max model length is 1024 for this model, although the actual embedding size for GPT small is 768
The beginning of sequence token <|startoftext|> token has the id 50257
The end of sequence token <|endoftext|> has the id 50256
The padding token <|pad|> has the id 50258
Thu Sep  1 08:38:35 2022
106,531 training samples
11,837 validation samples
Thu Sep  1 08:39:47 2022
Model loaded to GPU
Thu Sep  1 08:40:07 2022

======== Epoch 1 / 3 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.3727775812149048.   Elapsed: 0:03:17.
0: rootsYou can find your pick in Bombay, Kheda or Aligarh. The rest is right. The other two parts are the same. It may be that you should consider it and put up with it. This is what you should keep on asking. The first part is a lot more than a little easier. I feel that my hunger is as much as it is. My weight can grow more if I meet more of her. It can be that of any type, as it can get, but only if you can meet her. If you want to, I am better. We should only have a little time and rest for one hour, for that it means your weight will come in the train and you will be able to take the train. So long as the train is ready and that is for you. The next part is a little harder. One should start from here and come over by train. I do not need any more. In order to reach here, the
  Batch 2,000  of  26,633. Loss: 0.49371036887168884.   Elapsed: 0:06:42.
0:  aimsI have your letter of the 11th September. I am sending it all to Mr. Rustomjee. I have not received any reply, it is very good, it has been very good for you. It is because I believe it is necessary to do so. But you should not insist on any one of these reasons.
  Batch 3,000  of  26,633. Loss: 0.2933245301246643.   Elapsed: 0:10:06.
0: omanYour letter is a good letter. If you wish to send me a letter, I shall reply after your letter. I would like you to send me a detailed report of my opinion about what you expect to do in the matter. If you want me to do so, I do not wish to wait for the Viceroy or the Viceroy to come before you. I have received the letter. I hope you have accepted it. I am trying to make it clear to all the persons who have taken up such a course that I shall give you more time for the inquiry. I can say with the confidence that you will be able to come when you see me. I must say if you do not do, that I intend to come when I am on that train. But your mind is very weak. It will take your time to heal and then you have got the work done. The best thing is if you have your mind on the right path. I hope it will be through
  Batch 4,000  of  26,633. Loss: 0.5538762211799622.   Elapsed: 0:13:31.
0:  auditYou will come into being in the present when you are at the top of your business. And you will give to your friends, relatives and others the news about your work. My firmness in this manner is not due to the present. The work of your countrymen is not only to be appreciated, but it is not a good thing if you cannot be satisfied with your work. It is the Congress that has been working for the past many months. You know that the time for us is about the middle. You need not be indifferent to it. I want you to be content with your work. But then. What I want you to do by the end of this year.
  Batch 5,000  of  26,633. Loss: 0.2704867720603943.   Elapsed: 0:16:55.
0: heimerHere is the reply by Vakils to my letter, saying, “We shall take you here for further work during the coming month. I am very happy that you have the letter. You have taken me away for a long time. I shall have to wait a day before it becomes necessary to proceed to England, but I am sure you will find some work to be done.
  Batch 6,000  of  26,633. Loss: 0.8708100318908691.   Elapsed: 0:20:19.
0: ausibleIf I have the time, I would request you to give me one of the items. I want you to improve your health.
  Batch 7,000  of  26,633. Loss: 0.32363420724868774.   Elapsed: 0:23:42.
0:  soundThis is all the while trying to make one more suggestion, and I hope that it is no more than one from here to make one better and the second less important. I dont see that a large number of our women are not interested in this cause, which is why they are so. We do not want them to come to it as the girls of India, who were here and for whom we were born but of which God has a most important place, can remain there and do anything.
  Batch 8,000  of  26,633. Loss: 0.4614584147930145.   Elapsed: 0:27:06.
0: ánA gentleman from Bombay writes about the present position. “A person with the greatest experience does his task. He never goes to the place he likes.” It is very dangerous. There is another way which would be better. We should not be surprised if we are not taken seriously. At present we have not found any alternative; rather, the situation is a little difficult. Our capacity in the absence of any of the workers in that place is in perpetual difficulty. There was no question at all, because of that, but now what shall we do? It is our duty to give them every facility of service in this connection. There is only one solution.” He is now taking up other remedies. A worker is not in the wrong; the work in the right direction. He is quite capable of doing that. That he is capable of the service of his people and this is the reason why he is taking his work up by going to the place. For many days,
  Batch 9,000  of  26,633. Loss: 0.9255771636962891.   Elapsed: 0:30:32.
0: acitySpun 90 minutes for your weekly routine.
  Batch 10,000  of  26,633. Loss: 0.6591060757637024.   Elapsed: 0:33:55.
0: ottenI am sending the letter in reply to the post. The second and third is that it is my opinion that the fast should be taken up on the 4th at least. I have just got an order from Mr. Rustomjee and if it is the intention I can see there is no need to worry. What I am writing now would be no more than the fast, if it is taken up on a Friday, the 2nd on the 3rd and on the 5th on the 6th instant.
  Batch 11,000  of  26,633. Loss: 0.3243859112262726.   Elapsed: 0:37:20.
0: 146The writer of “Saroj” asks why, if the public has any knowledge of this thing, a particular community of Muslims has ever become a minority in this way. We do not know what kind of country we are. But when the Hindus, Hindus and Sikhs fight, they go the whole length of their lives without fear. In the same way, if we, then, take our own lives, we become free; as individuals we should have in ourselves been free from the fear of the police. If we do not believe in the non-co-operation and the mob, the law of non-co-operation and the law of non-co-operation is being obeyed in this manner; the Muslims, Christians and Parsis are ready, and so we would be ready to fight that, but this cannot be done away with. For the rest, I do not think a separate Hindu army, like mine, should go into the country. The Muslims have
  Batch 12,000  of  26,633. Loss: 0.44140100479125977.   Elapsed: 0:40:45.
0:  translatedSo the question of khaddar requires a change. This means not only to make the whole Khaddar a centre for economic progress but, instead of stopping the propaganda through which the work is being carried, the whole village need not starve without putting up a fine spinning-wheel or spinning-wheel for earning his livelihood. At the present moment I am giving an address in Bombay to the public in Bangalore, and I have suggested the following.
  Batch 13,000  of  26,633. Loss: 1.2146430015563965.   Elapsed: 0:44:11.
0:  WaldThe incident that occurred yesterday in the national capital shows how cruel this is, that no man, woman or child can be the best judge. It was very unfortunate for me to have been called to the meeting when the meeting was held. I have now learnt from the experience. The meeting, in my opinion, will have no bearing on my decision not to publish it now because of the circumstances of the marriage. It was a great surprise indeed that I was there when I got the news of the marriage, but that has made me sad. At the outset, my belief has been that the marriage seems to have no connection with the marriage of the girl. I am no stranger to it, no more an adviser than a good man.
  Batch 14,000  of  26,633. Loss: 0.3298036456108093.   Elapsed: 0:47:36.
0:  seasonThe Government of India has promised to take to their possession all the land owned by the people of this land. They are now paying heavily to the Government which is doing its work honestly and honestly. We want to have a clear understanding of the rights of British Indians in this Colony, our land, the law and order, the education of the villagers of that Colony, all this and so on; we want them to be treated equally as subjects of Government. We want them to be so as to help us. We want that they should be treated as our brothers and sisters. If, therefore, they wish to be free, that should be done by them as long as there is mutual consent.
  Batch 15,000  of  26,633. Loss: 0.7765573859214783.   Elapsed: 0:51:01.
0: 401I got your letter. You have also written in Hindi and English. The same thing was done here in Kashi. And now you have become quite well. God has saved us from the sin of drinking. In the past, all these things had gone on. We are all in the right place. You must have learnt from the experiences that I had heard about people doing all this work.
  Batch 16,000  of  26,633. Loss: 1.052242636680603.   Elapsed: 0:54:24.
0:  DominIn my view, the Congress itself is not yet at the point of a communal conflict and I have endeavoured to bring on its own a solution of the problems that existed in the Congress during the past ten years or even during the last twenty years, by the very nature and necessity I have endeavoured to bring before the country for consideration for the future generations of my life. I should therefore invite the attention of everyone to the importance of the Congress.
  Batch 17,000  of  26,633. Loss: 0.8759819865226746.   Elapsed: 0:57:52.
0:  henceI have your letter. Of course we need not complain about it. If the fast is confined only to one meal a day and you will not come out, I would advise you to take up spinning and go to bed. If the fast is intended to help one, ask me to do so. I will get the news. I shall see about it if I get the time.
  Batch 18,000  of  26,633. Loss: 0.26272448897361755.   Elapsed: 1:01:17.
0:  slidingYou do not find that you have to be cautious while taking an examination. Your reason is what God ordains. How can I give an assurance? I am sending my papers to you.
  Batch 19,000  of  26,633. Loss: 0.6615938544273376.   Elapsed: 1:04:42.
0:  GraceWhen I was in Burma I observed the same condition that I observed in Burma and Burma as in South Africa. How then would I? My country was like a mother to me, a mother to me and that was the thing you mentioned. But you did not do so and hence I had gone down there. If, however, I was an Englishman, I would feel deeply grieved at the sad state of things there.
  Batch 20,000  of  26,633. Loss: 0.6591067910194397.   Elapsed: 1:08:06.
0:  QueenslandWhen you come to Panchgani for the present, you will find many young men and women of these schools who are working together in the same kitchen. As the children are ill in health, you will observe brahmacharya, and this has only led to greater ones being devoted to their work.
  Batch 21,000  of  26,633. Loss: 0.3622573912143707.   Elapsed: 1:11:31.
0:  OrigIam sorry I forgot to write to you in my letter. I have no news about it yet. It is only a slight change from the original and it does not appear to me in your letter. I had intended to write a wire from Bombay. Now if I have to go there I shall write a brief letter to Pyarelal.
  Batch 22,000  of  26,633. Loss: 1.200019121170044.   Elapsed: 1:14:55.
0:  mandMy advice to you is, if you get it, we can have it safely through some other means.
  Batch 23,000  of  26,633. Loss: 0.14675480127334595.   Elapsed: 1:18:18.
0:  DeepThe work for the charkha, however, is becoming quite big. But since there is no end to work in hand, it will not make you lose heart. We shall have to see who goes on this march. When you are leaving us, there will be no end to work. What is my message to you, I wish your life would be a blessing in disguise. It is so well. Hence, as I am making you, I can say with all our great enthusiasm that there is nothing to do with this. You will also see how to set a limit to the extent you can. You will see that a man who goes in vain cannot take his place in our hearts. For any work that you have done does not become ones aim at any cost.
  Batch 24,000  of  26,633. Loss: 0.2943846583366394.   Elapsed: 1:21:43.
0:  defI understand about Ratilal. If I had heard something from you, you would have not seen it here.
  Batch 25,000  of  26,633. Loss: 0.20097878575325012.   Elapsed: 1:25:06.
0:  SarIf the Government does not listen to us, if we want swaraj, we shall have no objection to it, because if we want swaraj, they are our brothers. The Hindus and the Muslims are ours as brothers. The Government wants independence as soon as we have won independence. We have to fight on our own hands as an independent nation. In this respect, however, swaraj is a religious act. It does not involve political power politics, as they say, because they want to take advantage of the strength of reason.
  Batch 26,000  of  26,633. Loss: 0.5619271397590637.   Elapsed: 1:28:30.
0: vityThe following is a translation of what you have been able to say by the inmates, namely, “The Indians” is published in your issue of The Star. It is a translation of the whole thing. I suppose you have received the original, i.e., “The [Natal] Leader or “The Rand Daily Telegraph”. I have not read it in this way. The translation is not yet known. If they cannot, the Indians may revise the translations to suit the requirements, but I would like to say that it is wrong in that it was not possible. Even now, that the Transvaal [South African] Leader was given to us, we should only give the satisfaction that, in spite of such improvements, it was written by all Indians in the Transvaal. Now that there is some change of heart, we have to give the permission to the others to do so. We have therefore to give the assurance that if the Transva

  Average training loss: 0.57
  Training epoch took: 1:30:42
Thu Sep  1 10:10:49 2022

Running Validation...
  Validation Loss: 0.53
  Validation took: 0:03:11

======== Epoch 2 / 3 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.31986793875694275.   Elapsed: 0:03:18.
0:  subjectsI have read your speech. I hope the Congress Working Committee will let you know what the other parties have to say. If you can agree, you should make a representation to Lord Irwin and give expression to their grievances. My offer will be that they would make no public statement in effect, so that they can show to their countrymen what they really are. If that is the meaning of this letter I shall take it that those in whose opinion the resolution has been passed cannot possibly go, but they may come forward and make no proposals, as I have pointed out the reasons for the Congress and the other parties. To my mind the position here is only one—that is, the Congress. Therefore it would be wrong on my part not to accept even the minimum of the conditions for acceptance of the Congress creed, nor should we oppose a single resolution. You may also take it that I would not have accepted that resolution in any spirit.
  Batch 2,000  of  26,633. Loss: 0.5277178287506104.   Elapsed: 0:06:45.
0:  changingThe following interesting corollary to the note from Shrimati Dharmadashankar Gholam:
  Batch 3,000  of  26,633. Loss: 0.4013044536113739.   Elapsed: 0:10:09.
0: 253In my humble opinion the most striking feature of the struggle is the fact that the authorities did nothing to ensure success. It has no direct bearing on the people as to their status and position. When the Government has decided to arrest all satyagrahis, it is but natural. It is just as wrong to arrest anybody who has not only been arrested, but has committed a crime of an unprovoked nature.
  Batch 4,000  of  26,633. Loss: 0.6288009881973267.   Elapsed: 0:13:33.
0:  StandardsTo be satisfied at a place like Bombay is indeed very difficult. There were also meetings. There were a lot of people there, but the thing took up such a difficult time that I must not write.
  Batch 5,000  of  26,633. Loss: 0.499160498380661.   Elapsed: 0:16:57.
0:  tro3. The following are the first two: (a) The Hindu State is the only one, i.e., a Hindu, from among all communities—the majority being the Mussalman or a Hindu.
  Batch 6,000  of  26,633. Loss: 0.31342142820358276.   Elapsed: 0:20:22.
0: inchI do not know if it is a great thing indeed that you find a person like you all taking up a new profession and putting an end to all that has been going on for years. The man that is trying to gain a name for himself has become a gentleman who does not hesitate to speak out against you. His wife is also in the same position. Both the husband and wife have not realized the real worth of the name. That is what the best thing would be to have. But let us take up the service of the country because when you are free from your duty you will be a man of service to the nation.
  Batch 7,000  of  26,633. Loss: 0.7130310535430908.   Elapsed: 0:23:47.
0:  RoyalIf my advice is to be found from any other friend, it will be to consult him.
  Batch 8,000  of  26,633. Loss: 0.45082345604896545.   Elapsed: 0:27:11.
0:  essayIf the Congress has an anti-Indian spirit, it must be un-violence. The best way is, I believe, to seek an answer to the charge. The most effective course, in the long run, for the nation will be that they must be more gentle with the minority. I have not heard of any such thing in my life; I have never known it before. A Congress like the League or any other organization can not give any such advice. Every one of you must have thought of this as a new life, and it is also a matter of great humiliation for all of us to get excited about things. Our real test will be that we shall be convinced in the near future about any such thing. The whole burden of discipline has been attached to our shoulders. Our work, therefore, is not to do with an army or any other organization. That is the way. I do not understand. The new Government is one of anarchy. I want this much of independence.
  Batch 9,000  of  26,633. Loss: 0.38659095764160156.   Elapsed: 0:30:36.
0:  eatI have your letter. Since you had not received it I do not know the matter. You must have made up your mind to write to me. If you do not, I may write something. I have not yet got the letter you suggest for the sake of writing to me. You may now come whenever you wish to. In order to go there you should stay on and if possible look after your health. Write to me regularly. Ask Mahadev also whether you cannot spare yourself for the service. I dont want to make another suggestion for him. If you think it necessary to send you there, let him see what you want from me. Do come.
  Batch 10,000  of  26,633. Loss: 0.571908712387085.   Elapsed: 0:34:02.
0: phanA word or two from you. It would be no joke to make a mistake, if I could have avoided your writing. If there would be some mistake in the thought of writing this, why should you not give my word? What can I say about it in my letter of the 14th?
  Batch 11,000  of  26,633. Loss: 0.2549073100090027.   Elapsed: 0:37:28.
0:  TrainI have your letter. I should like you to come over whenever you wish to but this letter has no use for you. I would feel very sorry to read the article sent by you in Harijan. It could be, in fact, some time before we meet. But there are very few who have heard from you in this matter. We can go on as long as we have no fear whatever of the future. We can do this through our own work. God will see how you get out of it. We cannot go on like this. I shall await your reply when you come.
  Batch 12,000  of  26,633. Loss: 0.24915991723537445.   Elapsed: 0:40:52.
0:  adviserI got your letter. In spite of my advice I dont mind. I want to write this letter by the same post. You did well not to write about anything else. However, this is my only reply.
  Batch 13,000  of  26,633. Loss: 0.8472073078155518.   Elapsed: 0:44:17.
0: @@You have been asked to sign what is meant in your letter and send it. Your letter to my wife tells me that you are happy and happy. I know about the change. If it should do you think of the thing independently, I shall be able to decide further when you decide to read the letter.
  Batch 14,000  of  26,633. Loss: 0.3693886399269104.   Elapsed: 0:47:41.
0:  WyomingYou will have noticed that the Gujarati translation has been translated on the train. It was not intended to introduce the Gujarati edition. The work was carried on during the Congress week. It has been translated into Hindustani.
  Batch 15,000  of  26,633. Loss: 0.5214073657989502.   Elapsed: 0:51:05.
0: ULSo you see the question of swaraj in the form of your own swaraj. How much you want it from the masses? How will it be a matter of choice? How do we expect the Viceroy to take up this responsibility in a hurry. And why should we have the duty of collecting so much money and being able to collect this amount? What are we able to achieve? I can decide only after I receive what help the latter need.
  Batch 16,000  of  26,633. Loss: 0.27784913778305054.   Elapsed: 0:54:29.
0: �I do not know whether you will accept the resignation of Sardar Vallabhbhai. The same thing happened in Ahmedabad also. The postscript was a long one and I felt that the way of resigning should be given up. The way in which you did, as you have been doing, is quite the opposite of what I have been told. But Sardar Vallabhbhai is in no way my follower. He can take no one with him in his way. But we need not resort to civil disobedience against him. Even though he has no other remedy to offer, he can only be courteous to the Sardar and his colleagues. My way of doing it will depend on the manner in which he behaves at the present moment. I should have no quarrel with him and he should be the very personification for which we are talking. But this can only mean that, if we have got rid of the corruption that we are fighting in the name of sw
  Batch 17,000  of  26,633. Loss: 0.6354331374168396.   Elapsed: 0:57:53.
0: askI am glad you are having a chat with Mr. M. V. Menon, the Colonial Secretary, and others. I hope, as also Mr. Gokhale, who has very kindly attended to your inquiry as well as his previous letter, of the 10th instant, on the position taken up by the Congress.
  Batch 18,000  of  26,633. Loss: 0.3742370903491974.   Elapsed: 1:01:18.
0:  syrupWith reference to your letter of 31st October last, I have written a letter to the Viceroy in reply to your letter. There is not a trace left on your letter.
  Batch 19,000  of  26,633. Loss: 0.4193110167980194.   Elapsed: 1:04:41.
0:  MHMy Association sincerely regards this as a serious grievance and wants every member of the community to withdraw the letter addressed to him, and to approach the Government with any reasonable grievance which may be offered to him so as to make it clear that his case before the Board of Management is just and the Board of Management may not be allowed to intervene. The Board respectfully protests that the position of the management is entirely unchanged, and that, since this is my view, it is necessary to take steps for our success in the matter.
  Batch 20,000  of  26,633. Loss: 0.5652617812156677.   Elapsed: 1:08:05.
0: oysNow that you are coming here I shall give you an idea of things. I have been thinking for a long time about what to do in the case of the poor. But since your visit to Rajkot I know that it was your duty to make your heart strong. You should resolve the matter. After you reach Delhi, let me know.
  Batch 21,000  of  26,633. Loss: 0.6229895949363708.   Elapsed: 1:11:29.
0:  RibI had to thank my co-workers. They gave me the fullest satisfaction. My love was due to them and I hope they will not allow others to harm them. The real friend of India is the late Shrimati Natarajans wife and her family. Her character is her great-natured nature. The more the greater will be the service she rendered to society through her labour, the stronger will be the feeling. But that would be an additional merit. It is the greatest misfortune I possess in India.
  Batch 22,000  of  26,633. Loss: 0.6947505474090576.   Elapsed: 1:14:53.
0:  hedgeI would have been delighted if you had been here and had brought us two brothers, and there would have been no hitch if the boys and girls of Phoenix had come together. If you had told me that it would have been better if you had been here, I would certainly have known that the boys were from England, but you have also told me also that the girls were from the Ashram or from a school in the West. Yours is not the place where I am called. Yours has a few students who are not at all educated. But you should tell them that I have been trying to learn Urdu by heart. I am a very sincere worker and I have tried to teach a few to the boys of Phoenix, and as a result I get them useful training. As the boys get this training, I think we have a good opportunity for showing how to create real awakening among the girls.
  Batch 23,000  of  26,633. Loss: 0.36134257912635803.   Elapsed: 1:18:17.
0:  wedI have received complaints about your fever. I must have heard of your having been ill. Please tell us what it means to get cured [of it]. You should be good enough to know what the remedies are. In the first place, the first thing to do is to keep your tongue and limbs healthy. Write to Dr. Roy about all that. He should be given all the facilities he needs. Keep on writing to me.
  Batch 24,000  of  26,633. Loss: 0.6859543919563293.   Elapsed: 1:21:41.
0:  HolyI would gladly publish this statement without any hesitation, since my personal connection with Mr. Hosken was intimately connected with the movement for the redress of the grievances of the people concerned. The writer is not aware of the great grievances of the local residents of Natal. This pamphlet is an attempt to understand the grievances of the British Indian traders. It gives further information. It is an attempt to see how the new law might be enforced. I have never been able to find that a single Indian has been in the country under indenture with the result that he is deprived of his rights. In my humble opinion, however, they are nothing less than an attempt to create a disorganized society and to put forth a system of government. In every province, there is no sense of justice in giving a cause. There are hundreds of Indians, not more than six thousand, in India who cannot afford to pay this tax. That they are poor or otherwise, can give no comfort to the poor who can
  Batch 25,000  of  26,633. Loss: 0.33111920952796936.   Elapsed: 1:25:06.
0:  dismayKrishnadas is going to Ahmedabad. He will give you your address to the Harijan Sevak Sangh. If he does, send him to Wardha. He will then address to the meeting in Wardha. I shall let you know when it will be time for you to come here. There are many women also in the Ashram. The chief reason for this is that there are fewer women in the Ashram. They are also working hard.
  Batch 26,000  of  26,633. Loss: 0.5339946150779724.   Elapsed: 1:28:30.
0:  evenlyI have your letter. I am surprised that you will not get my letter. I must write to you. You must learn the art of writing and writing. There should be no difficulty about the writing.

  Average training loss: 0.52
  Training epoch took: 1:30:41
Thu Sep  1 11:44:41 2022

Running Validation...
  Validation Loss: 0.52
  Validation took: 0:03:11

======== Epoch 3 / 3 ========
Training...
  Batch 1,000  of  26,633. Loss: 0.277262806892395.   Elapsed: 0:03:18.
0:  satisfI got your letter. What about the spinning-wheel? The spinning-wheel cannot be kept alive because everyone can ply it without help. There are quite a few books here.
  Batch 2,000  of  26,633. Loss: 0.4236743152141571.   Elapsed: 0:06:43.
0:  SourceThere is no cause for worry. My decision to go to Bihar was against the interests of the Hindus. The Muslims also had the same objection. The Muslims wanted to bring Pakistan out. But they have been partitioned. No, we shall meet in Allahabad. If we cannot achieve this, they will forget it. How can we get rid of that? How can we, then, meet Pakistan with such a sense of our responsibility that they do not have any alternative but to accept us as our brothers? In the circumstances this is not at all advisable. We should be brave if the Congress is to become free. The Muslim League is a political organization. They may be able to wrest power in the province. In all this, we have to develop the capacity for the freedom of both Hindus and Muslims. That, however, cannot be done before we arrive here.
  Batch 3,000  of  26,633. Loss: 0.22891117632389069.   Elapsed: 0:10:08.
0:  limIt does not appear that there are now men who have not paid their wages and even as many are still awaiting the payment of its revenue dues, there is nothing to worry about. It should be realized that the payment made to this Government is not adequate or a cause of grievance.
  Batch 4,000  of  26,633. Loss: 0.6004093885421753.   Elapsed: 0:13:31.
0:  toutedThe new Bill, which was introduced, now lapses under the late Sir Raymond Wests signature. It could not possibly be considered in question, and the object of the new Bill was to raise the Indian franchise. The question of immigration was always raised, and then, Mr. Gandhi asked, and that the Transvaal [passenger] Bill was, in my opinion, absolutely an Act. After I had discussed the above Bill with General Smuts, I pointed out to him that it would be difficult to grant a vote to the Bill even if the amendment was withdrawn by consent. It was no longer necessary that the Bills be revised. Now, too, the Immigration Bill was introduced; and I suggested to the Minister of the Interior that the amendment should not be made. The amendment of the Bill was then carried out. The amendment would be so amended that the provision regarding the issue would be considerably improved. This Bill is not intended for those Indians who were domiciled in the Colony
  Batch 5,000  of  26,633. Loss: 0.8679662346839905.   Elapsed: 0:16:56.
0:  CeltThe Congress is doing well to set out with the constructive programme. We should also have constructive activities with full understanding with and without the slightest doubt with the Government.
  Batch 6,000  of  26,633. Loss: 0.8571312427520752.   Elapsed: 0:20:20.
0:  meetsA friend has sent me your letter. I am writing this letter on my way back from Delhi. I shall reach there on Wednesday at about nine oclock. It is now nine oclock. I am to leave Delhi on Saturday.
  Batch 7,000  of  26,633. Loss: 0.689114511013031.   Elapsed: 0:23:44.
0:  translationThe same applies to Harijans too. Not to the Harijans will you agree? It is certainly true. But if it was impossible, why could we not discuss the matter with the Harijans because they were so few? And why should we not discuss the problem at all? What can we expect if we are unable to understand the meaning of the word ‘Harijan? What can we do when someone suggests that they should do something?
  Batch 8,000  of  26,633. Loss: 0.3899286687374115.   Elapsed: 0:27:08.
0:  usualBut when there is a change, it can be proved only through the Press. A man who has no time will not be able to write to me; nor, of course, he will not be able to write, because there will be no harm in giving him something and his mind will be disturbed by it. In spite of the change, we shall never find any difficulty in understanding the meaning.
  Batch 9,000  of  26,633. Loss: 0.6294029951095581.   Elapsed: 0:30:32.
0: DamageI got your letter. I am sorry I forgot your word. But why should we be upset even when somebody asks for you to be a child? You have made such a mistake and now it is for you to decide what you should do. I do hope you have come to an understanding with the Viceroy that I should act in your own good. Let me know your position as regards a girl.
  Batch 10,000  of  26,633. Loss: 0.40873050689697266.   Elapsed: 0:33:57.
0:  instrumentsIn my humble opinion, the above statement has been made under an unanswerable circumstance, namely, in support of my claim for responsible government. But since it will be an entirely unintended thing for me to take any action on my behalf, I have accepted the assurance given by the M.A. of course. As I am now on a tour, the time for coming here may be. I wish I could at the same time send you the Viceroys message and invite the Viceroys guidance. And I hope, of course, that those who have doubts will be able to see me. I hope, of course, that you do not think that I am responsible for the happenings in London which I can no longer attend, or that, if that is required, you may take them on record. I shall know when you finally meet me.
  Batch 11,000  of  26,633. Loss: 0.4443233907222748.   Elapsed: 0:37:22.
0:  grainThe letter to Chhaganlal regarding the book I had to write to you was dated the 15th instant and I had no difficulty about understanding its contents. I wrote yesterday to the Editor the following morning:
  Batch 12,000  of  26,633. Loss: 0.35419338941574097.   Elapsed: 0:40:46.
0:  MongWhat is required, however, is a complete change of heart among Congress workers. It would be a farce. But no one can say why it should not be there. It was a bad augury that I should retire from active politics. However, so far as I am concerned, I should work under the guidance and guidance of the Congress. At this stage of my life, I have decided to retire from politics. I have kept myself in touch with the local authorities. I have no quarrel with them; I can serve them better. I had never claimed to represent them. In the eyes of the Congress I do so in the hope of winning them over. But what is more, my services are still going on in the work that I do. Therefore I am no longer interested in making the people forget me. My love for them is ever with them. My advice has always been, “Please remember me as your friend and guide and serve me as your companion. Do not
  Batch 13,000  of  26,633. Loss: 0.5126112103462219.   Elapsed: 0:44:12.
0:  dismayI got your letter. It was the best thing you could do for the Ashram. I shall be happy if you come and take me if I go somewhere. If I have to do it, I have to come. You must come immediately. The main thing is that whatever work we get here will be done by our own labour.
  Batch 14,000  of  26,633. Loss: 0.996166467666626.   Elapsed: 0:47:38.
0: acesI must ask myself whether, even if one step is taken, I would have made one change and the other would have been taken in the course of what has been called, and what is now. As I have said above, even though one step has no significance, I know that I have been making some changes during my incarceration. I now observe from the foregoing that, if I can achieve much in this task, I shall be able to achieve nothing more in this life. But if I am right in my view, I may have to decide, if I cannot, what to do now.
  Batch 15,000  of  26,633. Loss: 0.6593640446662903.   Elapsed: 0:51:02.
0:  MYIn regard to my letter of the 25th January last, which I read in your last letter, I would like to make a brief history of the work, and, so long as I do not know the facts of the case in my mind, I should be glad to find out where the statements made in my letter are true. I take as a matter of fact as I can think of them and their value. I do not at all like the Government doing the wrong. I consider it to be my duty to give some reason for the refusal of my permission to pass any such Bill, and so I have no hesitation in advising the Government to introduce the Bill.
  Batch 16,000  of  26,633. Loss: 0.39263415336608887.   Elapsed: 0:54:29.
0:  lawmakersThe first question, as has been stated in the preface in the first paragraph of this paragraph, is, if there are any such disputes, the right of refusal to admit the minor children to a Harijan school in accordance with the scheme laid down therein, and the question of giving them education in Harijan will remain.
  Batch 17,000  of  26,633. Loss: 0.6760329604148865.   Elapsed: 0:57:54.
0:  SSDGandhiji and the people of the place are passing through a difficult time. He has given me a good account of it.
  Batch 18,000  of  26,633. Loss: 0.6334731578826904.   Elapsed: 1:01:21.
0: ablishmentI am anxious about the good work. I have not seen the copy of Harilals article on swadeshi. My impression is that this is written in haste. I have gone to Bombay and so have Sjt. Satish Babu. But I see no difficulty in sending you the whole of your book to him. I do hope your health is all right.
  Batch 19,000  of  26,633. Loss: 0.2487851083278656.   Elapsed: 1:04:46.
0:  layoutAs to the Transvaal, my answer is that the Indians in the Transvaal are not aware of the Indian question and have not been able to find out for themselves the facts. The question is very vital. I have, therefore, written to you, in the early stages, what is to happen. I thank you once more for your letter of the 23rd ultimo.
  Batch 20,000  of  26,633. Loss: 0.4330776035785675.   Elapsed: 1:08:12.
0:  SedMy last letter was received on the 4th. I am sorry about the absence of your brother. He should not have troubled you, he should have given us a notice to leave this place and take the risk of having to die. The doctor does not seem to have examined me. But he says that I have become weak and that he has come back. I have given him the news about Sushilabehn. I could see her very clearly. He is in Calcutta, and will go a few days yet. It is true that she has recovered, but she has lost her health. There is no question of her living long. I have, therefore, decided to write to her, and see what can be done if she wishes to. I also see what should be done if she cannot restrain herself. I feel that Sushila and I may have more time today and we shall discuss the matter after we reach Wardha.
  Batch 21,000  of  26,633. Loss: 0.25756675004959106.   Elapsed: 1:11:40.
0: consMy answer was that he had a talk with me. He was talking to me when he said that some sort of settlement was not possible. He had said that he would not discuss the matter further if I could be present. I told him that if he could not agree, I would put forward the proposal of a arbitration. He is also a man of learning. He has got no time to write to me. But if I do it, I am afraid the question will be raised in the Press.
  Batch 22,000  of  26,633. Loss: 0.45388856530189514.   Elapsed: 1:15:05.
0:  reconciliationLet this be my first question and also the first. You will tell me when I arrive there and when I return it will be your first question. If you have not, dont you have the time? There is certainly some talk going on. I am not at all perturbed.
  Batch 23,000  of  26,633. Loss: 0.6558365821838379.   Elapsed: 1:18:29.
0: GIThe only way to save our cattle is to provide them facilities for food. A carpenter would not require the services of a paid employee because the farmer would not be able to get a decent supply of milk and, when one finds that there are in the villages plenty of cattle available, a well-organized man would not need even a goat. Moreover, he is obliged if the goat cannot protect itself against being destroyed.
  Batch 24,000  of  26,633. Loss: 0.6731799244880676.   Elapsed: 1:21:56.
0:  tangI can understand all the grief caused by your death. For I do not want you to grieve over your deaths. May God preserve you. I wish you long life to your work. I wish you would realize that you are a dear friend of mine, a source of strength for me, and also the best of all.
  Batch 25,000  of  26,633. Loss: 0.29150673747062683.   Elapsed: 1:25:22.
0:  marijuanaIt is not right that a young man should remain confined to bed for long but, when he feels well-deserved, he goes on trying his level best to see that his mind is fully occupied in his own work and is free from desire and anger. The body is as well kept, the mind is too light and the soul is too weak to keep it from being tossed. But this is not how the body is to behave. It seems to have lost its strength. We have been saying, “We can do the work which we can do.” So to us, if we feel that it is our dharma to do it, we must follow the dharma of dharma. In fact, the body is a part of our dharma. If it refuses to do that, even then, it should be regarded as the fruit of our efforts to attain it. If our body is weak, we should not feel the pain that we experience from the loss of body
  Batch 26,000  of  26,633. Loss: 0.43229103088378906.   Elapsed: 1:28:50.
0: MANI do not understand how you are going to defend yourself. I understand the position you have taken up regarding your diet in the Navajivan case. I shall be satisfied if you send me one line of demerits from here. You must have received a list of the books. The books are not being read out to me. I shall be ina position to take them up.

  Average training loss: 0.50
  Training epoch took: 1:31:01
Thu Sep  1 13:18:52 2022

Running Validation...
  Validation Loss: 0.51
  Validation took: 0:03:11

Training complete!
Total training took 4:41:56 (h:mm:ss)
Thu Sep  1 13:22:03 2022
       Training Loss  Valid. Loss Training Time Validation Time
epoch                                                          
1               0.57         0.53       1:30:42         0:03:11
2               0.52         0.52       1:30:41         0:03:11
3               0.50         0.51       1:31:01         0:03:11
The GPT-2 model has 148 different named parameters.

==== Embedding Layer ====

transformer.wte.weight                                  (50259, 768)
transformer.wpe.weight                                   (1024, 768)

==== First Transformer ====

transformer.h.0.ln_1.weight                                   (768,)
transformer.h.0.ln_1.bias                                     (768,)
transformer.h.0.attn.c_attn.weight                       (768, 2304)
transformer.h.0.attn.c_attn.bias                             (2304,)
transformer.h.0.attn.c_proj.weight                        (768, 768)
transformer.h.0.attn.c_proj.bias                              (768,)
transformer.h.0.ln_2.weight                                   (768,)
transformer.h.0.ln_2.bias                                     (768,)
transformer.h.0.mlp.c_fc.weight                          (768, 3072)
transformer.h.0.mlp.c_fc.bias                                (3072,)
transformer.h.0.mlp.c_proj.weight                        (3072, 768)
transformer.h.0.mlp.c_proj.bias                               (768,)

==== Output Layer ====

transformer.ln_f.weight                                       (768,)
transformer.ln_f.bias                                         (768,)
Saving model to /media/data_dump/Ritwik/ggpt/model_save/
Model and tokenizer loaded!
Thu Sep  1 13:22:08 2022
tensor([[50257,   314,  4601,   284,   910,   326]], device='cuda:0')
0:  I wish to say that for our present conduct we cannot do without khadi. We may use the latter only if it can be had from a distance and thus not only for a short period.


1:  I wish to say that such a statement is not necessary for the object of the Congress to show that I personally did nothing to incite public opinion against it. This was wholly untrue. It was intended that I would be prepared to undergo the sufferings of my opponent if they could not possibly do so. But as for the Congress, I said once more that in my opinion the Congress had the whole thing in its consideration, and that the Swarajists should offer me the due use. This was not in fact my position, and therefore it was clear that the Congress would not give up the struggle. It is in such a matter that my fast was a breach of the vow of non-violence.


2:  I wish to say that we need not have a great crowd at the stations as we should have. At all stations it is unnecessary for an important station like Bombay.


Thu Sep  1 13:22:10 2022