Hugging Face's logo
Eugene Siow's picture
15 3 233

Eugene Siow

eugenesiow
Mi6paulino's profile picture silka's profile picture darkzbaron's profile picture
Ā·
https://eugenesiow.com

AI & ML interests

None yet

Recent Activity

liked a model 5 days ago
Datou1111/shou_xin
upvoted an article 12 days ago
šŸŗšŸ¦ā€ā¬› LLM Comparison/Test: 25 SOTA LLMs (including QwQ) through 59 MMLU-Pro CS benchmark runs
Reacted to m-ric's post with šŸ”„ 12 days ago
š—¦š—µš—¼š˜„š—Øš—œ: š—® š˜€š—ŗš—®š—¹š—¹ š—²š—»š—±-š˜š—¼-š—²š—»š—± š—®š—“š—²š—»š˜ š˜š—µš—®š˜ š—°š—®š—» š—»š—®š˜ƒš—¶š—“š—®š˜š—² š—®š—»š˜† š—Øš—œ š—®š—»š—± š—¼š˜‚š˜š—½š—²š—暝—³š—¼š—暝—ŗš˜€ š—ŗš˜‚š—°š—µ š—Æš—¶š—“š—“š—²š—æ š˜€š˜†š˜€š˜š—²š—ŗš˜€! šŸ“² A team from NUS and Microsoft just released an agent that can act on any UI (Desktop, Android, Web) without needing additional text information. It works extremely well : they applied their method on a tiny Qwen2-VL-2B, and they managed to beat methods that use either much more powerful vision models (like GPT-4V) without using any additional info (e.g. leveraging the DOM of a webpage) like previous methods did ! šŸ‘šŸ‘ They started from the idea that most existing methods rely heavily on text, which makes them less generalizable, while letting aside rich UI structure that user actually rely on when navigating this interfaces. āš™ļø They put several good ideas to work: šŸ’” Simplify screenshots to the max: They prune a lot the heavy visual content of UI screenshots, by removing cloned image patches (like any vast patch of the same color will be reduced to a small patch, while maintaining positional embeddings), then group patches from the same GUI elements together to simplify even further šŸ’” Build a truly generalist dataset: To train a general UI agent, you need trajectories from each possible UI, and express them in a common language. Authors merge datasets like OmniAct for Desktop, Mind2Web for websites, AMEX for Android trajectories to create a high-quality and diverse dataset. āž”ļø Nice results ensued: They fine-tune a tiny Qwen-2-VL-2B on their method, and it reaches SOTA on several task (element identification, web navigation), even beating methods that either use additional info from the DOM or use much bigger VLMS like GPT-4v! šŸ† And performance could certainly jump with a slightly bigger vision model. Let's hope the community builds this soon! šŸš€ Paper added to my "Agents" collection šŸ‘‰ https://huggingface.co/collections/m-ric/agents-65ba776fbd9e29f771c07d4e
View all activity

Organizations

Spaces-explorers's profile picture DSO National Laboratories's profile picture AI Singapore's profile picture DSO Intern's profile picture Top Contributors: Model Downloads's profile picture open/ acc's profile picture

eugenesiow's activity

New activity in eugenesiow/remove-bg over 1 year ago

Update requirements.txt
#2 opened over 1 year ago by itacaiunas
New activity in HuggingFaceH4/starchat-alpha over 1 year ago

Model doesn't end/terminate generation: Need to modify EOS token

9
#2 opened over 1 year ago by eugenesiow
New activity in OpenAssistant/oasst-sft-6-llama-30b-xor over 1 year ago

Chat/Conversation Templates

1
#9 opened over 1 year ago by eugenesiow
New activity in eugenesiow/bart-paraphrase over 1 year ago

How to add the number of sequences in model.generate for Bart Paraphrase

1
#1 opened over 2 years ago by pratikkotian04

Adding `safetensors` variant of this model
#2 opened almost 2 years ago by SFconvertbot
New activity in eugenesiow/Div2k about 2 years ago

Why lr and hr is string not image?

1
#3 opened about 2 years ago by Freed-Wu
New activity in eugenesiow/BSD100 about 2 years ago

Fix task tags
#4 opened about 2 years ago by albertvillanova
New activity in eugenesiow/PIRM about 2 years ago

Fix task_ids
#2 opened about 2 years ago by albertvillanova
New activity in eugenesiow/Div2k about 2 years ago

Fix task_ids
#2 opened about 2 years ago by albertvillanova
New activity in eugenesiow/BSD100 about 2 years ago

Fix language and license tag names
#1 opened about 2 years ago by albertvillanova
New activity in eugenesiow/Set14 about 2 years ago

Fix task_ids
#2 opened about 2 years ago by albertvillanova
New activity in eugenesiow/Set5 about 2 years ago

Fix task_ids
#1 opened about 2 years ago by albertvillanova
New activity in eugenesiow/Urban100 about 2 years ago

Fix task_ids
#2 opened about 2 years ago by albertvillanova
New activity in eugenesiow/Urban100 over 2 years ago

Fix `license` metadata
#1 opened over 2 years ago by julien-c
New activity in eugenesiow/PIRM over 2 years ago

Fix `license` metadata
#1 opened over 2 years ago by julien-c
New activity in eugenesiow/bart-paraphrase over 2 years ago

How to add the number of sequences in model.generate for Bart Paraphrase

1
#1 opened over 2 years ago by pratikkotian04