SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
Model Sources
Model Labels
| Label |
Examples |
| 1 |
- '4 Hours before takeoff we received a Mail stating a cryptic message that there are disruptions to be expected as there is a limit on how many planes can leave at the same time. So did the capacity of the Heathrow Airport really hit British Airways by surprise, 4h before departure? Anyhow - we took the one hour delay so what - but then we have been forced to check in our Hand luggage. I travel only with hand luggage to avoid waiting for the ultra slow processing of the checked in luggage. Overall 2h later at home than planed, with really no reason, just due to incompetent people. Service level far worse then Ryanair and triple the price. Really never again. Thanks for nothing.'
- 'Our connecting flight from London to Glasgow was canceled and we were forced to wait 12 hours in the airport for a new flight. When we received our luggage at Glasgow we discovered that our luggage was soaking wet. My wife’s luggage contents were wet and had to be air dried.'
- "Absolutely the worst experience ever. Flew into Heathrow, no problem. Waited for 5 hours, three extra hours, for our connection to Porto then the cancelled the flight. Scrambled for a hotel and a new flight the next day. They routed us through Madrid the next morning. That flight was 1.5 hours late so we missed our connection to Porto. Because we missed the connection (their fault), they cancelled our return flights. After two hours on the phone with customer service from the Madrid airport, they booked us a new return flight, then they made is pay $ 1300 to fly from Madrid to Porto. Then, for no reason they cancelled our return flight again! After hours, and hours and hours with customer service, we finally got a return flight five days after our original return flight. Meanwhile, they lost my wife's luggage. We finally got 15 DAYS after they lost it. Return flight? More problems with our reservations. At no point was British Airways helpful. They left the onus on us to fix their mistakes at every turn. They completely ruined a three week vacation and they don't care. They wouldn't even let us speak to a supervisor or manager. Awful, bordering on passenger abuse."
|
| 4 |
- 'I'm sure that BA have gradually made their economy experience (BA Euroflyer out of Gatwick) so miserable that passengers are tempted to upgrade to the lacklustre and overpriced business class. Long wait at the biz checkin ( due to status ) and dealt with by a bored agent - OK third party, but still representing BA - who hardly spoke and avoided eye contact and was more focussed on her fingernails. Lounge was crowded and grubby but at least the staff there are pleasant and helpful On board the seats are cramped and uncomfortable and it took over two hours before crew appeared with food sales and the "free water" which on Gatwick services consists of a half full plastic cup rather than the luxury of a small bottle.. A fellow passenger mentioned the delay and the fact that they had no sandwiches for sale which prompted the flight attendant to say that the service was much better on Easyjet who do two service runs etc.'
- 'BA standards continue to decline every time I fly with them. This time, a 45 min late departure, which seems the norm for BA, and no information from the rude and clueless boarding gate staff. The Club lounge was overcrowded, dirty and grubby and cleaning staff could not cope. How many more cutbacks is BA going to apply before they become truly low cost. As a 35 year veteran user of BA, while people complained of Alex Cruz cutting costs, the past 3 years has seen them decline very quickly under their latest CEO. One reads the occasional article about BA improving things, but I have yet to see anything substantive. There seem to be few British staff left amongst the cabin crew, so maybe time for a name change and give the flag carrier status to a proper low-cost - and sadly the staff service has declined year on year. Onboard, the catering remains poor and surprised that Do&Co can produce such rubbish meals when I have sampled their Turkish Airlines equivalent in August that was superb - guessing BA are too tight with their budget. Onboard WiFi service is a joke and should be avoided.'
- 'Good lounge at Cape Town. On time departure. Drinks high quality. Sound dinner service. Breakfast reasonable. Arrival lounge was good but showers cold.'
|
| 5 |
- "Despite boarding being the usual free for all at LHR with groups 1-3 being called to board all at once. Whilst those passengers who had agreed to check in a bag in at the gate, getting to board before anyone else! However, the service on board was impeccable. Superb crew, friendly, attentive, interacting with passengers and generally appearing to be enjoying their job. Like BA in the old days and at its best. Food choice and quality seems to have improved too (no beef cheeks thankfully) and even the Club kitchen appears to have returned. Old style Club seating, but is at least better, and feels less cramped, on the 787's."
- "Flew back from Malta after scattering our son's ashes at his favourite spots. When we boarded the aircraft for our return flight my wife became upset. A member of the cabin crew noticed this and immediately spoke to my wife and tried to reassure her. Her name is Tracey, an exceptional crew member. I noticed she spoke to other travellers in a courteous manner and was friendly. Each time she passed my wife she asked how she was and would she like anything. This kindness made a difficult time easier for us both. Aircraft seats were comfortable and service good for a 2hrs. 45mins. flight. Thank you B.A."
- 'Excellent service levels, proactive crew and superb food and beverages. I found all aspects of the service to be superior to BA business class, even the new BA Club Suites. First Class felt exclusive, the dedicated crew were clearly used to working the premium cabin, enjoying their work, were proactive, friendly, very respectful and the food and beverage offering a considerable step up from business class. The seat, with sliding double doors was spacious, well designed and very comfortable. British Airways offered a solid First Class service, from Concorde Lounge to boarding (which was actually by group number for a change) to the very warm and genuine welcome on boarding. A consistent, solid, comfortable service from start to finish and if the price was right, I would not hesitate to book First Class with British Airways again.'
|
| 2 |
- "I take comfort in reading the last ten or so reviews to see I'm not the only traveller to put in words, how absolutely abysmal British Airways are! The only reason we are stuck using this awful airline is to use up points. We booked Club Europe months ago. London to Athens, sat in row 3 and told they had run out of food options, only choice was lentil pie or green curry. Returning Athens to London Club Europe. Couldn't check in online as promised so arrived at the airport 3hrs early. To be told they had overbooked the flight and basically we were being 'bumped' to economy. This airline fails to deliver time and time again. It is a national disgrace and I implore everyone to choose an alternative. "
- "You can buy sandwiches and crisps but don't expect a hot meal. Because the plane is a narrow body , there are no screens. Not helping matters is the fact that the BA prebooking shop for food didn't recognise the flight number, and the onboard WiFi didn't work. "
- 'The airplanes and the lounges are worn out, old and broken. From Dallas to Heathrow, multiple first class seating and electronics were non functional and poorly designed. The first class seating from Heathrow to Dubrovnik was nothing more than a tray blocking the middle seat of a standard coach ticket. Very disappointed.'
|
| 3 |
- 'We traveled to Lisbon for our dream vacation, a cruise to Portugal and Spain. Our friends did not EVER get their luggage. It was a two week cruise. Two weeks without a change of clothes or her CPAP machine. Contacting customer service was a nightmare. We never talked to a real person. Very little effort was put into getting them their bags. In one port we were on the ship and the bags were at the airport. The airlines did not deliver the bags nor did they tell our friends that they needed to go get them. The airline couldn’t be bothered to take the bags from the airport to the ship. BA says it is their policy to get you your bags within 72 hours. That is a joke! It’s been over two weeks and they still don’t have them. They are back home in the US and last they heard the bags were in Lisbon! If your have a choice, do not fly British Airways. Customer Service is non existent!'
- "I have flow on BA several times since the airline started non-stop service between Austin, TX and London Heathrow. Every year the service gets worse. I brought a tape measure with me to see how far apart the seats are in the economy section. 25½ inches. BA has removed bathrooms in order to squeeze a few more rows of seats onto the aircraft. Thankfully I'm not a big person and was able to squeeze into my seat without being too uncomfortable. When BA first started flying out of Austin, the food was great. Now, when I arrive at Heathrow, I get real food and take it on the plane with me. I don't know where or who makes BA's food, but it is not eatable. If BA did not have a direct flight from my hometown to London, I would fly with a different airline. When we checked into Heathrow, the line was extremely long and only one person was checking people in. It took nearly an hour to get to the counter. I understand airlines are making record profits and need to make up lost revenue from the pandemic, but I think BA has gone too far and has little concern for customer service in pursuit of profits."
- 'Angry, disappointed, and unsatisfied. My route was from London to Atlanta. My suitcase was not boarded, therefore not landed with me. For both comfort and safety reason, a bag always fly with its passenger and that did not happen. Claims and few phone calls were made by desk assistants who answered my questions unprofessionally and miserably. Certainly, I was left with nothing but my backpack which contained not more than few snacks. Neither clothes nor anything else was ever provided as an apology. Meanwhile, I was also told that my bag would have been delivered through the next 24 hours which also did not happen. British Airways is a great airline to fly with but its organization, when it comes to customer service, is poor and uncertain. Still waiting for my bag.'
|
| 6 |
- 'They downgraded me from business to premium economy. They took three months to get back to me and offered me 200 pounds or 400 AUD. The difference between business class and premium economy (which I never fly) is $3000 AUD and the difference between economy and business is $6500 AUD. They owe me at least 1500 pounds and to be very fair 3250 pounds. The downgrade was insulting and incredibly painful as I had a recent cancer operation and have a back injury which I have pointed out to them - I need to lie down. The wait for them to get around to me has been numbing. The insult of their offer has been the slap in the face to continue my degradation. The people I have dealt with at the airport have been down to earth, honest hard working people. The actual staff on the plane were warm and friendly - and helpful. The systems under which BA operates are some sort of medieval torture.'
- 'BA cancelled our flight and could not book us onto any flight for the next 4 days. I had to spend £5k in expenses to get us back with another airline. 2 months later I am still waiting for reimbursement and have had no response to my 3 written complaints. Even the call centre staff tell me that head office have taken down the telephone number and email for the department who deal with this, so even they cannot escalate or inquire. Utterly useless.\n'
- "This is the worst experience I have ever had with any airline. I flew with British Airline on August 21st 2022. I had one checked bag that was lost at Heathrow Airport. I am aware that at that time there was a system error with the baggage belt and many travelers were affected. I filed a report for my missing bag on the same day I arrived at LAX airport, and that was on August 21st. I waited patiently for three weeks (which is the time frame to either find the bag or be eligible for a compensation). When I did not receive any update on my lost bag, I filed a claim for compensation, and that was on September 11th. Since then, I have been waiting for a response from the customer relation with no good signs of resolving the issue. I emailed them and tried to update the claim but still no response until today. I contacted the baggage team and they were not helpful and did not ease my prolonged waiting. The only way to contact customer relations is via the website and they're not even getting back to me. Every time I talk to a customer service agent, they tell me to wait some more. I have been waiting for almost three months now and my issue is not resolved yet. This whole experience was a hassle. Very unprofessional for a renowned airline like BA."
|
Evaluation
Metrics
| Label |
Accuracy |
| all |
0.8333 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("setfit_model_id")
preds = model("I flew from Istanbul to London in Business class. For more than half of the flight a child was watching an iPad very loud with no phones plugged in - it was three rows in front of me and disrupted my flight as I could hear it above my headphones. Staff thought this was fine and refused to tell the mom to turn it down. Why is this acceptable on a flight especially in business class? Has this airline turned into a zoo? Are staff given no training in customer service? Why is a child allowed to disrupt other passengers? When I told the staff she just laughed at me. BA customer service is a disgrace. I've not even mentioned the food but it was bad, really bad.")
Training Details
Training Set Metrics
| Training set |
Min |
Median |
Max |
| Word count |
23 |
144.6667 |
418 |
| Label |
Training Sample Count |
| 1 |
7 |
| 2 |
7 |
| 3 |
7 |
| 4 |
7 |
| 5 |
7 |
| 6 |
7 |
Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 123
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
| Epoch |
Step |
Training Loss |
Validation Loss |
| 0.0054 |
1 |
0.1708 |
- |
| 0.2717 |
50 |
0.0981 |
- |
| 0.5435 |
100 |
0.0739 |
- |
| 0.8152 |
150 |
0.0039 |
- |
| 1.0870 |
200 |
0.0005 |
- |
| 1.3587 |
250 |
0.0007 |
- |
| 1.6304 |
300 |
0.0002 |
- |
| 1.9022 |
350 |
0.0005 |
- |
| 2.1739 |
400 |
0.0004 |
- |
| 2.4457 |
450 |
0.0003 |
- |
| 2.7174 |
500 |
0.0005 |
- |
| 2.9891 |
550 |
0.0004 |
- |
Framework Versions
- Python: 3.8.10
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.37.2
- PyTorch: 2.2.0
- Datasets: 2.17.1
- Tokenizers: 0.15.2
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
