Nicolay Rusnachenko

nicolay-r

AI & ML interests

Information Retrieval・Medical Multimodal NLP (🖼+📝) Research Fellow @BU_Research・software developer http://arekit.io・PhD in NLP

Organizations

None yet

Posts 28

view post
Post
129
📢 Have you ever been wondered how specifically Transformers were capable for handling long input contexts?
I got a chance to tackle this through long document texts summarization problem, and delighted to share the related survey and diagram for a quick skimming below:

Preprint 📝 https://nicolay-r.github.io/website/data/preprint-AINL_2023_longt5_summarization.pdf
Springer 📝 https://link.springer.com/article/10.1007/s10958-024-07435-z

🎯 The aim of the survey was the development of the long-document summarizer for mass-media news in Vietnamese language. 🇻🇳

Sharing for a quick skimming of the methods performance overview of various LM-based solution across several datasets, covering domain-oriented advances in Vietnamese language (see attached screenshots)

As for solution we consider:
☑️ 1. Adapt existed google/pegasus-cnn_dailymail for summarizing large dataset for arranging training
☑️ 2. Tuning google/long-t5-tglobal-large suitable for performing generative summarization.

Implementation details:
🌟 https://github.com/nicolay-r/ViLongT5
(Simplier to go with huggingface rather flaxformer that so far become a legacy engine)
view post
Post
1766
📢 If you're aimed at processing complex dependencies in spreadsheet data with LLM Chain-of-Thought technique, then this update might be valuable for you 💎

The updated 📦 bulk-chain-0.24.1 which is aimed at iterative processing of CSV/JSONL data with no-string dependencies from third party LLM frameworks is out 🎉

📦: https://pypi.org/project/bulk-chain/0.24.1/
🌟: https://github.com/nicolay-r/bulk-chain
📘: https://github.com/nicolay-r/bulk-chain/issues/26

The key feature of bulk-chain is SQLite caching that saves your time ⏰️ and money 💵 by guarantee no-data-lost, which is important once using the remote LLM providers such as OpenAI, ReplicateIO, OpenRouter, etc.

🔧 This release has the following updates:
✅ Improved stability for various header conditions and the related support from SQLite
✅ Manual setup for ID column / assigning the ID
✅ Make CSV-related setups dynamic, that refers to the related Python 📦 csv package.

Quick start on GoogleColab:
📙: https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb

Below is an example of the three simple steps in pictures:
1. ⬇️ Package installation
2. ✍️ Declaring schema
3. 🚀 Launching inference for your data with Replicate and 🤖 meta-llama/Llama-3.1-405B

datasets

None public yet