January 2023

The Smart Dev Approach to Startup R&D

⋅

The Smart Dev Approach to Startup R&D For seven years, I’ve been helping dozens of companies in various stages research new domains, improve sales and technical culture, build new products (including data-driven and machine learning solutions), and succeed in fundraising. In this post, I want to crystallize an approach that successful startups leverage when working with outside consultants in their research phase. I call this approach “smart dev”. Like “smart money”, which is VC money invested by firms […]

Ver mais

Liked Liked

The Transformer Family Version 2.0

⋅

Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — restructure the hierarchy of sections and improve many sections with more recent papers. Version 2.0 is a superset of the old version, about twice the length. Notations Symbol Meaning $d$ The model size / hidden state dimension / positional encoding size. $h$ The number of […]

Ver mais

Liked Liked
ChatGPT: A Game-Changer in the World of AI?

ChatGPT: A Game-Changer in the World of AI?

⋅

ChatGPT, a chatbot capable of conducting conversations in a human-like manner, has made headlines both in specialized technology publications and mainstream news. Some experts predict that this marks the beginning of a new era in AI. The president and co-founder of OpenAI, Greg Brockman, expressed his excitement about the release of ChatGPT by tweeting: “Prediction: 2023 will make 2022 look like a sleepy year for AI advancement & adoption.” Some experts call the bot “the killer of Google” […]

Ver mais

Liked Liked
Introduction to Generative AI

Introduction to Generative AI

⋅

Generative artificial intelligence has seen an incredible popularity surge in 2022. Big Think has called it ‘the technology of the year’, and judging from the amount of attention and VC support generative AI startups have been gaining this year, this claim is more than justified. Moreover, tech experts say that in the next few years, not only will the development of generative AI not slow down but will also rapidly increase, conquering new and new fields. In this […]

Ver mais

Liked Liked

Curated Resources and Trustworthy Experts: The Key Ingredients for Finding Accurate Answers to Technical Questions in the Future

⋅

Conversational chat bots such as ChatGPT probably will not be able replace traditional search engines and expert knowledge anytime soon. With the vast amount of misinformation available on the internet, the ability to distinguish between credible and unreliable sources remains challenging and crucial.

Ver mais

Liked Liked

Large Transformer Model Inference Optimization

⋅

[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful transformer for solving real-world tasks at scale. Why is it hard to run inference for large transformer models? Besides the increasing size of SoTA models, there are […]

Ver mais

Liked Liked
K-Means Clustering in Machine Learning

K-Means Clustering in Machine Learning

⋅

K-means is a data clustering approach for unsupervised machine learning that can separate unlabeled data into a predetermined number of disjoint groups of equal variance – clusters – based on their similarities. It’s a popular algorithm thanks to its ease of use and speed on large datasets. In this blog post, we look at its underlying principles, use cases, as well as benefits and limitations. What is k-means clustering? K-means is an iterative algorithm that splits a dataset […]

Ver mais

Liked Liked

Get Started with Rust: Generics

⋅

Get Started with Rust: Generics Generic programming allows programmers to write general algorithms that work with arbitrary types. It reduces code duplication and provides type safety, which enables us to write more concise and clean code. This approach is also known as parametric polymorphism or templates in other languages. Rust supports two types of generic code: Compile-time generics, similar to C++ templates. Run-time generics, similar to virtual functions in C++ and generics in Java. Serokell has a broad […]

Ver mais

Liked Liked

↑

Digitado © 2025