2-finetuned-xlm-r-masakhaner-swa-whole-word-phonetic

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.0492

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-08
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	61	39.7928
No log	2.0	122	39.8195
No log	3.0	183	39.8228
No log	4.0	244	39.0793
No log	5.0	305	38.6628
No log	6.0	366	37.4014
No log	7.0	427	36.8203
No log	8.0	488	36.2607
41.0436	9.0	549	35.9316
41.0436	10.0	610	34.3282
41.0436	11.0	671	33.3604
41.0436	12.0	732	32.4470
41.0436	13.0	793	31.5403
41.0436	14.0	854	30.9665
41.0436	15.0	915	29.4171
41.0436	16.0	976	28.4589
32.5506	17.0	1037	27.2699
32.5506	18.0	1098	26.0614
32.5506	19.0	1159	25.1529
32.5506	20.0	1220	24.3080
32.5506	21.0	1281	23.1511
32.5506	22.0	1342	22.3332
32.5506	23.0	1403	21.7154
32.5506	24.0	1464	20.7367
24.3212	25.0	1525	20.1804
24.3212	26.0	1586	19.5591
24.3212	27.0	1647	18.8760
24.3212	28.0	1708	18.3291
24.3212	29.0	1769	18.0296
24.3212	30.0	1830	17.4091
24.3212	31.0	1891	17.2150
24.3212	32.0	1952	16.6483
18.5662	33.0	2013	16.2697
18.5662	34.0	2074	16.1714
18.5662	35.0	2135	15.8008
18.5662	36.0	2196	15.2647
18.5662	37.0	2257	15.4195
18.5662	38.0	2318	14.9898
18.5662	39.0	2379	14.8599
18.5662	40.0	2440	14.6119
15.1141	41.0	2501	14.4644
15.1141	42.0	2562	14.3398
15.1141	43.0	2623	14.4107
15.1141	44.0	2684	13.9039
15.1141	45.0	2745	14.0043
15.1141	46.0	2806	13.9534
15.1141	47.0	2867	13.9267
15.1141	48.0	2928	13.5888
15.1141	49.0	2989	13.6313
13.182	50.0	3050	13.5405
13.182	51.0	3111	13.3378
13.182	52.0	3172	13.3160
13.182	53.0	3233	13.1356
13.182	54.0	3294	13.2483
13.182	55.0	3355	13.1259
13.182	56.0	3416	13.1775
13.182	57.0	3477	13.1118
12.1712	58.0	3538	12.9363
12.1712	59.0	3599	12.8765
12.1712	60.0	3660	12.7923
12.1712	61.0	3721	12.9732
12.1712	62.0	3782	12.8606
12.1712	63.0	3843	12.7897
12.1712	64.0	3904	12.6516
12.1712	65.0	3965	12.6242
11.5853	66.0	4026	12.5954
11.5853	67.0	4087	12.4505
11.5853	68.0	4148	12.5595
11.5853	69.0	4209	12.4287
11.5853	70.0	4270	12.4768
11.5853	71.0	4331	12.4003
11.5853	72.0	4392	12.4323
11.5853	73.0	4453	12.4179
11.25	74.0	4514	12.3767
11.25	75.0	4575	12.4342
11.25	76.0	4636	12.2565
11.25	77.0	4697	12.3128
11.25	78.0	4758	12.3749
11.25	79.0	4819	12.3565
11.25	80.0	4880	12.3288
11.25	81.0	4941	12.2014
11.0038	82.0	5002	12.1772
11.0038	83.0	5063	12.2690
11.0038	84.0	5124	12.1987
11.0038	85.0	5185	12.0654
11.0038	86.0	5246	12.1295
11.0038	87.0	5307	12.1560
11.0038	88.0	5368	11.9662
11.0038	89.0	5429	12.1134
11.0038	90.0	5490	12.0294
10.8283	91.0	5551	12.1112
10.8283	92.0	5612	12.0910
10.8283	93.0	5673	12.0087
10.8283	94.0	5734	11.9803
10.8283	95.0	5795	11.9094
10.8283	96.0	5856	12.1592
10.8283	97.0	5917	11.9361
10.8283	98.0	5978	11.9596
10.693	99.0	6039	11.9026
10.693	100.0	6100	12.0040
10.693	101.0	6161	11.8866
10.693	102.0	6222	11.9536
10.693	103.0	6283	11.8034
10.693	104.0	6344	11.6885
10.693	105.0	6405	11.8505
10.693	106.0	6466	11.8280
10.5875	107.0	6527	11.7874
10.5875	108.0	6588	11.7348
10.5875	109.0	6649	11.7765
10.5875	110.0	6710	11.7527
10.5875	111.0	6771	11.6816
10.5875	112.0	6832	11.7396
10.5875	113.0	6893	11.6475
10.5875	114.0	6954	11.7010
10.5114	115.0	7015	11.7049
10.5114	116.0	7076	11.7967
10.5114	117.0	7137	11.7248
10.5114	118.0	7198	11.6549
10.5114	119.0	7259	11.5194
10.5114	120.0	7320	11.6924
10.5114	121.0	7381	11.5194
10.5114	122.0	7442	11.6607
10.4791	123.0	7503	11.5189
10.4791	124.0	7564	11.5525
10.4791	125.0	7625	11.5226
10.4791	126.0	7686	11.4954
10.4791	127.0	7747	11.5944
10.4791	128.0	7808	11.7056
10.4791	129.0	7869	11.7464
10.4791	130.0	7930	11.4851
10.4791	131.0	7991	11.3106
10.4223	132.0	8052	11.6615
10.4223	133.0	8113	11.5130
10.4223	134.0	8174	11.5866
10.4223	135.0	8235	11.4999
10.4223	136.0	8296	11.5736
10.4223	137.0	8357	11.6101
10.4223	138.0	8418	11.6018
10.4223	139.0	8479	11.5062
10.3725	140.0	8540	11.4695
10.3725	141.0	8601	11.5745
10.3725	142.0	8662	11.2131
10.3725	143.0	8723	11.4861
10.3725	144.0	8784	11.3521
10.3725	145.0	8845	11.4302
10.3725	146.0	8906	11.3891
10.3725	147.0	8967	11.4948
10.3579	148.0	9028	11.4140
10.3579	149.0	9089	11.2809
10.3579	150.0	9150	11.5084
10.3579	151.0	9211	11.4942
10.3579	152.0	9272	11.4316
10.3579	153.0	9333	11.5054
10.3579	154.0	9394	11.4354
10.3579	155.0	9455	11.3708
10.3411	156.0	9516	11.6017
10.3411	157.0	9577	11.4415
10.3411	158.0	9638	11.6150
10.3411	159.0	9699	11.3789
10.3411	160.0	9760	11.3342
10.3411	161.0	9821	11.4761
10.3411	162.0	9882	11.3000
10.3411	163.0	9943	11.4109
10.3236	164.0	10004	11.4250
10.3236	165.0	10065	11.3250
10.3236	166.0	10126	11.4247
10.3236	167.0	10187	11.1662
10.3236	168.0	10248	11.4494
10.3236	169.0	10309	11.3871
10.3236	170.0	10370	11.2961
10.3236	171.0	10431	11.3576
10.3236	172.0	10492	11.4350
10.3059	173.0	10553	11.3612
10.3059	174.0	10614	11.3979
10.3059	175.0	10675	11.3716
10.3059	176.0	10736	11.4017
10.3059	177.0	10797	11.5342
10.3059	178.0	10858	11.2274
10.3059	179.0	10919	11.4326
10.3059	180.0	10980	11.4779
10.2637	181.0	11041	11.3424
10.2637	182.0	11102	11.2459
10.2637	183.0	11163	11.3178
10.2637	184.0	11224	11.3254
10.2637	185.0	11285	11.2635
10.2637	186.0	11346	11.2145
10.2637	187.0	11407	11.3280
10.2637	188.0	11468	11.3373
10.2837	189.0	11529	11.1808
10.2837	190.0	11590	11.2220
10.2837	191.0	11651	11.1251
10.2837	192.0	11712	11.3569
10.2837	193.0	11773	11.1888
10.2837	194.0	11834	11.2572
10.2837	195.0	11895	11.3158
10.2837	196.0	11956	11.2948
10.2404	197.0	12017	11.2186
10.2404	198.0	12078	11.2823
10.2404	199.0	12139	11.2506
10.2404	200.0	12200	11.5542
10.2404	201.0	12261	11.3140
10.2404	202.0	12322	11.2008
10.2404	203.0	12383	11.1571
10.2404	204.0	12444	11.2337
10.2304	205.0	12505	11.2708
10.2304	206.0	12566	11.3081
10.2304	207.0	12627	11.1103
10.2304	208.0	12688	11.1240
10.2304	209.0	12749	11.3450
10.2304	210.0	12810	11.0590
10.2304	211.0	12871	11.2573
10.2304	212.0	12932	11.2076
10.2304	213.0	12993	11.1813
10.2405	214.0	13054	11.3382
10.2405	215.0	13115	11.4096
10.2405	216.0	13176	11.1521
10.2405	217.0	13237	11.3125
10.2405	218.0	13298	11.1917
10.2405	219.0	13359	11.2792
10.2405	220.0	13420	11.1236
10.2405	221.0	13481	11.2397
10.2096	222.0	13542	11.1875
10.2096	223.0	13603	11.3117
10.2096	224.0	13664	11.1565
10.2096	225.0	13725	11.4165
10.2096	226.0	13786	11.1474
10.2096	227.0	13847	10.9854
10.2096	228.0	13908	11.1346
10.2096	229.0	13969	11.2123
10.1998	230.0	14030	11.2454
10.1998	231.0	14091	11.3353
10.1998	232.0	14152	11.3052
10.1998	233.0	14213	11.1773
10.1998	234.0	14274	11.1327
10.1998	235.0	14335	11.3109
10.1998	236.0	14396	11.1788
10.1998	237.0	14457	11.3767
10.1947	238.0	14518	11.2157
10.1947	239.0	14579	11.2102
10.1947	240.0	14640	11.1842
10.1947	241.0	14701	11.1392
10.1947	242.0	14762	11.1399
10.1947	243.0	14823	11.1630
10.1947	244.0	14884	11.1972
10.1947	245.0	14945	11.0548
10.1922	246.0	15006	11.1279
10.1922	247.0	15067	11.0969
10.1922	248.0	15128	11.2348
10.1922	249.0	15189	11.1151
10.1922	250.0	15250	11.5368
10.1922	251.0	15311	11.2244
10.1922	252.0	15372	11.2102
10.1922	253.0	15433	11.2735
10.1922	254.0	15494	11.3227
10.1994	255.0	15555	11.2377
10.1994	256.0	15616	11.2835
10.1994	257.0	15677	11.3475
10.1994	258.0	15738	11.2092
10.1994	259.0	15799	11.1936
10.1994	260.0	15860	11.0319
10.1994	261.0	15921	11.1916
10.1994	262.0	15982	11.1357
10.1883	263.0	16043	10.9732
10.1883	264.0	16104	11.1839
10.1883	265.0	16165	11.0701
10.1883	266.0	16226	11.1613
10.1883	267.0	16287	11.1302
10.1883	268.0	16348	11.0951
10.1883	269.0	16409	11.0579
10.1883	270.0	16470	11.1459
10.1863	271.0	16531	11.1969
10.1863	272.0	16592	11.1275
10.1863	273.0	16653	11.1115
10.1863	274.0	16714	11.1285
10.1863	275.0	16775	11.1053
10.1863	276.0	16836	11.0105
10.1863	277.0	16897	11.1378
10.1863	278.0	16958	11.0771
10.1614	279.0	17019	11.0620
10.1614	280.0	17080	10.9906
10.1614	281.0	17141	11.0771
10.1614	282.0	17202	11.0357
10.1614	283.0	17263	11.0417
10.1614	284.0	17324	11.0287
10.1614	285.0	17385	11.1172
10.1614	286.0	17446	10.9257
10.1717	287.0	17507	11.2312
10.1717	288.0	17568	11.3800
10.1717	289.0	17629	11.1386
10.1717	290.0	17690	11.1724
10.1717	291.0	17751	11.1628
10.1717	292.0	17812	11.1226
10.1717	293.0	17873	11.1955
10.1717	294.0	17934	11.1388
10.1717	295.0	17995	11.0874
10.1806	296.0	18056	11.0813
10.1806	297.0	18117	11.1475
10.1806	298.0	18178	11.1678
10.1806	299.0	18239	11.2450
10.1806	300.0	18300	11.1936
10.1806	301.0	18361	11.1021
10.1806	302.0	18422	11.1874
10.1806	303.0	18483	11.1719
10.1683	304.0	18544	11.1554
10.1683	305.0	18605	11.0771
10.1683	306.0	18666	11.0676
10.1683	307.0	18727	11.1280
10.1683	308.0	18788	11.0236
10.1683	309.0	18849	11.1440
10.1683	310.0	18910	11.1843
10.1683	311.0	18971	11.0474
10.1437	312.0	19032	11.0391
10.1437	313.0	19093	10.9148
10.1437	314.0	19154	11.0575
10.1437	315.0	19215	11.1955
10.1437	316.0	19276	11.0053
10.1437	317.0	19337	11.0810
10.1437	318.0	19398	11.1360
10.1437	319.0	19459	11.2291
10.1539	320.0	19520	11.0239
10.1539	321.0	19581	11.1216
10.1539	322.0	19642	11.2516
10.1539	323.0	19703	10.9759
10.1539	324.0	19764	11.0398
10.1539	325.0	19825	11.0431
10.1539	326.0	19886	10.9151
10.1539	327.0	19947	11.0905
10.1432	328.0	20008	11.0099
10.1432	329.0	20069	11.0893
10.1432	330.0	20130	11.1344
10.1432	331.0	20191	11.0682
10.1432	332.0	20252	10.9558
10.1432	333.0	20313	11.0669
10.1432	334.0	20374	11.0556
10.1432	335.0	20435	11.2097
10.1432	336.0	20496	11.0200
10.1343	337.0	20557	10.9683
10.1343	338.0	20618	10.9824
10.1343	339.0	20679	11.1563
10.1343	340.0	20740	11.1489
10.1343	341.0	20801	11.1389
10.1343	342.0	20862	11.1128
10.1343	343.0	20923	11.0437
10.1343	344.0	20984	11.1005
10.143	345.0	21045	11.1696
10.143	346.0	21106	11.1356
10.143	347.0	21167	11.0798
10.143	348.0	21228	10.9183
10.143	349.0	21289	11.0879
10.143	350.0	21350	10.9651
10.143	351.0	21411	11.0724
10.143	352.0	21472	11.0264
10.1456	353.0	21533	11.1398
10.1456	354.0	21594	11.2497
10.1456	355.0	21655	10.8898
10.1456	356.0	21716	10.9631
10.1456	357.0	21777	11.0734
10.1456	358.0	21838	11.1226
10.1456	359.0	21899	11.1686
10.1456	360.0	21960	11.0314
10.1345	361.0	22021	11.0940
10.1345	362.0	22082	10.9222
10.1345	363.0	22143	11.1036
10.1345	364.0	22204	11.1905
10.1345	365.0	22265	10.9741
10.1345	366.0	22326	10.9092
10.1345	367.0	22387	11.0564
10.1345	368.0	22448	11.0534
10.1354	369.0	22509	11.0307
10.1354	370.0	22570	11.1469
10.1354	371.0	22631	11.0560
10.1354	372.0	22692	11.0240
10.1354	373.0	22753	10.9869
10.1354	374.0	22814	11.0004
10.1354	375.0	22875	11.1373
10.1354	376.0	22936	11.0955
10.1354	377.0	22997	11.0542
10.1382	378.0	23058	10.9813
10.1382	379.0	23119	10.9874
10.1382	380.0	23180	10.9736
10.1382	381.0	23241	11.1295
10.1382	382.0	23302	10.8724
10.1382	383.0	23363	10.9367
10.1382	384.0	23424	11.0516
10.1382	385.0	23485	11.0275
10.1246	386.0	23546	11.0184
10.1246	387.0	23607	11.0570
10.1246	388.0	23668	11.0246
10.1246	389.0	23729	11.0131
10.1246	390.0	23790	11.0168
10.1246	391.0	23851	11.0817
10.1246	392.0	23912	10.8949
10.1246	393.0	23973	10.7698
10.1173	394.0	24034	11.0041
10.1173	395.0	24095	10.9257
10.1173	396.0	24156	10.9295
10.1173	397.0	24217	10.9476
10.1173	398.0	24278	11.0583
10.1173	399.0	24339	11.0006
10.1173	400.0	24400	10.9714
10.1173	401.0	24461	11.0480
10.1253	402.0	24522	11.0213
10.1253	403.0	24583	10.9636
10.1253	404.0	24644	10.9886
10.1253	405.0	24705	11.0664
10.1253	406.0	24766	11.0462
10.1253	407.0	24827	11.0122
10.1253	408.0	24888	10.8572
10.1253	409.0	24949	11.1382
10.1386	410.0	25010	11.0700
10.1386	411.0	25071	10.9676
10.1386	412.0	25132	11.1865
10.1386	413.0	25193	11.0785
10.1386	414.0	25254	11.0290
10.1386	415.0	25315	11.1383
10.1386	416.0	25376	11.1139
10.1386	417.0	25437	11.0185
10.1386	418.0	25498	11.0187
10.1491	419.0	25559	11.0893
10.1491	420.0	25620	11.0348
10.1491	421.0	25681	10.9932
10.1491	422.0	25742	11.0765
10.1491	423.0	25803	11.0488
10.1491	424.0	25864	11.0241
10.1491	425.0	25925	11.0695
10.1491	426.0	25986	10.8986
10.1184	427.0	26047	10.8433
10.1184	428.0	26108	10.8476
10.1184	429.0	26169	10.9747
10.1184	430.0	26230	10.9259
10.1184	431.0	26291	10.8647
10.1184	432.0	26352	11.0280
10.1184	433.0	26413	10.9582
10.1184	434.0	26474	10.9810
10.1396	435.0	26535	11.0491
10.1396	436.0	26596	11.0700
10.1396	437.0	26657	10.9878
10.1396	438.0	26718	10.9400
10.1396	439.0	26779	10.8682
10.1396	440.0	26840	10.9667
10.1396	441.0	26901	11.0117
10.1396	442.0	26962	11.0374
10.1337	443.0	27023	11.1337
10.1337	444.0	27084	10.9415
10.1337	445.0	27145	11.0174
10.1337	446.0	27206	11.0239
10.1337	447.0	27267	10.8979
10.1337	448.0	27328	10.9217
10.1337	449.0	27389	10.8926
10.1337	450.0	27450	11.1219
10.1168	451.0	27511	10.8931
10.1168	452.0	27572	11.0112
10.1168	453.0	27633	10.9823
10.1168	454.0	27694	11.1091
10.1168	455.0	27755	10.8694
10.1168	456.0	27816	10.9625
10.1168	457.0	27877	10.8553
10.1168	458.0	27938	10.9888
10.1168	459.0	27999	10.9296
10.1229	460.0	28060	10.8895
10.1229	461.0	28121	10.9803
10.1229	462.0	28182	11.0144
10.1229	463.0	28243	11.0257
10.1229	464.0	28304	10.9141
10.1229	465.0	28365	11.1346
10.1229	466.0	28426	11.0254
10.1229	467.0	28487	11.0384
10.1179	468.0	28548	10.8491
10.1179	469.0	28609	11.0469
10.1179	470.0	28670	10.9678
10.1179	471.0	28731	10.8902
10.1179	472.0	28792	10.9649
10.1179	473.0	28853	10.9252
10.1179	474.0	28914	11.1131
10.1179	475.0	28975	11.0267
10.1189	476.0	29036	10.8428
10.1189	477.0	29097	11.0314
10.1189	478.0	29158	11.0936
10.1189	479.0	29219	10.9968
10.1189	480.0	29280	10.8721
10.1189	481.0	29341	11.0153
10.1189	482.0	29402	11.1761
10.1189	483.0	29463	10.9840
10.1153	484.0	29524	10.9648
10.1153	485.0	29585	11.1140
10.1153	486.0	29646	11.0212
10.1153	487.0	29707	10.9197
10.1153	488.0	29768	10.9798
10.1153	489.0	29829	10.9047
10.1153	490.0	29890	11.0146
10.1153	491.0	29951	11.0575
10.1141	492.0	30012	11.0400
10.1141	493.0	30073	11.0898
10.1141	494.0	30134	10.9910
10.1141	495.0	30195	11.0579
10.1141	496.0	30256	10.8580
10.1141	497.0	30317	10.9450
10.1141	498.0	30378	11.0523
10.1141	499.0	30439	11.1228
10.1176	500.0	30500	11.0492

Framework versions

• Transformers 4.20.1
• Pytorch 1.11.0
• Datasets 2.1.0
• Tokenizers 0.12.1