Developer News
Ask LukeW: PDF Parsing with Vision Models
Over the years, I've given more than 300 presentations on design. Most of these have been accompanied by a slide deck to illustrate my points and guide the narrative. But making the content in these decks work well with the Ask Luke conversational interface on this site has been challenging. So now I'm trying a new approach with AI vision models.
To avoid application specific formats (Keynote, PowerPoint), I've long been making my presentation slides available for download as PDF documents. These files usually consist of 100+ pages and often don't include a lot of text, leaning instead on visuals and charts to communicate information. To illustrate, here's of few of these slides from my Mind the Gap talk.
In an earlier article on how we built the Ask Luke conversational interface, I outlined the issues with extracting useful information from these documents. I wanted the content in these PDFs to be available when answering people's design questions in addition to the blog articles, videos and audio interviews that we were already using.
But even when we got text extraction from PDFs working well, running the process on any given PDF document would create many content embeddings of poor quality (like the one below). These content chunks would then end up influencing the answers we generated in less than helpful ways.
To prevent these from clogging up our limited context (how much content we can work with to create an answer) with useless results, we set up processes to remove low quality content chunks. While that improved things, the content in these presentations was no longer accessible to people asking questions on Ask Luke.
So we tried a different approach. Instead of extracting text from each page of a PDF presentation, we ran it through an AI vision model to create a detailed description of the content on the page. In the example below, the previous text extraction method (on the left) gets the content from the slide. The new vision model approach (on the right) though, does a much better job creating useful content for answering questions.
Here's another example illustrating the difference between the PDF text extraction method used before and the vision AI model currently in use. This time instead of a chart, we're generating a useful description of a diagram.
This change is now rolled out across all the PDFs the Ask Luke conversational interface can reference to answer design questions. Gone are useless content chunks and there's a lot more useful content immediately available.
Thanks to Yangguang Li for the dev help on this change.
<p>CSS Meditation #8:<code> .work +
CSS Meditation #8: .work + .life { border: 10px solid #000; }
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #7: Nobody is perf
CSS Meditation #7: Nobody is perf-ect.
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #6: The color space
CSS Meditation #6: The color space is always calc(rgb(0 255 0)+er) on the other side of the fence.
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #5: <code>:where(:is(
CSS Meditation #5: :where(:is(.my-mind))
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #4: Select, style,
CSS Meditation #4: Select, style, adjust. Select, style, adjust. Select, sty…
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #3: A pseudo is as a
CSS Meditation #3: A pseudo is as a pseudo does.
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #2: Who gives a
CSS Meditation #2: Who gives a flying frick what constitutes a “programming” language.
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
<p>CSS Meditation #1: If the code works
CSS Meditation #1: If the code works as expected and it fits your mental model, then it’s perfect.
originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
Ask LukeW: Text Generation Differences
As the number of highly capable large language models (LLMs) released continues to quickly increase, I added the ability to test new models when they become available in the Ask Luke conversational interface on this site.
For context there's a number of places in the Ask Luke pipeline that make use of AI models to transform, clean, embed, retrieve, generate content and more. I put together a short video that explains how this pipeline is constructed and why if you're interested.
Specifically for the content generation step, once the right content is found, ranked, and assembled into a set of instructions, I can select which large language model to send these instructions to. Every model gets the same instructions unless they can support a larger context window. In which case they might get more ranked results than a model with a smaller context size.
Despite the consistent instructions, switching LLMs can have a very big impact on answer generation. I'll leave you to guess which of these two answers is powered by OpenAI's GPT-4 and which one comes from Antrhopic's new (this week) Claude 3.5 Sonnet.
Some of you might astutely point out that the instruction set could be altered in specific ways when changing models. Recently, we've found the most advanced LLMs to be more interchangeable than before. But there's still differences in how they generate content as you can clearly see in the example above. Which one is best though... could soon be a matter of personal preference.
Thanks to Yangguang Li and Sam for the dev help on this feature.
CSS Container Queries
Container queries are often considered a modern approach to responsive web design where traditional media queries have long been the gold standard — the reason being that we can create layouts made with elements that respond to, say, the width of their containers rather than the width of the viewport.
.parent { container-name: hero-banner; container-type: inline-size; /* or container: hero-banner / inline-size; */ } } .child { display: flex; flex-direction: column; } /* When the container is greater than 60 characters... */ @container hero-banner (width > 60ch) { /* Change the flex direction of the .child element. */ .child { flex-direction: row; } } Why care about CSS Container Queries?- When using a container query, we give elements the ability to change based on their container’s size, not the viewport.
- They allow us to define all of the styles for a particular element in a more predictable way.
- They are more reusable than media queries in that they behave the same no matter where they are used. So, if you were to create a component that includes a container query, you could easily drop it into another project and it will still behave in the same predictable fashion.
- They introduce new types of CSS length units that can be used to size elements by their container’s size.
- Registering Elements as Containers
- Querying a Container
- Container Queries Properties & Values
- Specification
- Browser Support
- Demos!
- Articles & Tutorials
- References
This example registers a new container named card-grid that can be queried by its inline-size, which is a fancy way of saying its “width” when we’re working in a horizontal writing mode. It’s a logical property. Otherwise, “inline” would refer to the container’s “height” in a vertical writing mode.
- The container-name property is used to register an element as a container that applies styles to other elements based on the container’s size and styles.
- The container-type property is used to register an element as a container that can apply styles to other elements when it meets certain conditions.
- The container property is a shorthand that combines the container-name and container-type properties into a single declaration.
- The container-name property is optional. An unnamed container will match any container query that does not target a specific container, meaning it could match multiple conditions.
- The container-type property is required if we want to query a container by its size or inline-size. The size refers to the container’s inline or block direction, whichever is larger. The inline-size refers to the container’s width in the default horizontal writing mode.
- The container-type property’s default value is normal. And by “normal” that means all elements are containers by default, only they are called Style Containers and can only be queried by their applied styles. For example, we can query a container’s background-color value and apply styles to other elements when the value is a certain color value.
- A container cannot change its own styles. Rather, they change the styles of their contents instead. In other words, we cannot change the container’s background-color when it is a certain size — but we can change the background-color of any element inside the container. “You cannot style what you query” is a way to think about it.
- A container cannot be sized by what’s in it. Normally, an element’s contents influence its size — as in, the more content in it, the larger it will be, and vice versa. But a container must be sized explicitly as part of a flex or grid layout.
- The @container at-rule property informs the browser that we are working with a container query rather than, say, a media query (i.e., @media).
- The my-container part in there refers to the container’s name, as declared in the container’s container-name property.
- The article element represents an item in the container, whether it’s a direct child of the container or a further ancestor. Either way, the element must be in the container and it will get styles applied to it when the queried condition is matched.
- The container’s name is optional. If we leave it out, then any registered container would match when the conditions are met.
- A container’s width can be queried with when the container-type property is set to either size or inline-size. That’s because size can query the element’s width or height; meanwhile, inline-size can only refer to the width.
- You can query any length. So, in addition to width (i.e., inline-size), there’s an element’s aspect-ratio, block-size (i.e., height), and orientation (e.g. portrait and landscape).
- Queries support the range syntax. Most of the examples so far have shown “greater than” (>) and “less than” (<), but there is also “equals” (=) and combinations of the three, such as “more than or equal to” (>=) and “less than or equal to” (<=).
- Queries can be chained. That means we can write queries that meet multiple conditions with logical keywords, like and, or, and not.
- none: The element does not have a container name. This is true by default, so you will likely never use this value, as its purpose is purely to set the property’s default behavior.
- <custom-ident>: This is the name of the container, which can be anything, except for words that are reserved for other functions, including default, none, at, no, and or. Note that the names are not wrapped in quotes.
- Initial value: none
- Applies to: All elements
- Inherited: No
- Percentages: N/A
- Computed value: none or an ordered list of identifiers
- Canonical order: Per grammar
- Animation: Not animatable
- normal: This indicates that the element is a container that can be queried by its styles rather than size. All elements are technically containers by default, so we don’t even need to explicitly assign a container-type to define a style container.
- size: This is if we want to query a container by its size, whether we’re talking about the inline or block direction.
- inline-size: This allows us to query a container by its inline size, which is equivalent to width in a standard horizontal writing mode. This is perhaps the most commonly used value, as we can establish responsive designs based on element size rather than the size of the viewport as we would normally do with media queries.
- Initial value: normal
- Applies to: All elements
- Inherited: No
- Percentages: N/A
- Computed value: As specified by keyword
- Canonical order: Per grammar
- Animation: Not animatable
If <'container-type'> is omitted, it is reset to its initial value of normalwhich defines a style container instead of a size container. In other words, all elements are style containers by default, unless we explicitly set the container-type property value to either size or inline-size which allows us to query a container’s size dimensions.
Open in Almanac- Initial value: none / normal
- Applies to: All elements
- Inherited: No
- Percentages: N/A
- Computed value: As specified
- Canonical order: Per grammar
- Animation: Not animatable
Container Style Queries is another piece of the CSS Container Queries puzzle. Instead of querying a container by its size or inline-size, we can query a container’s CSS styles. And when the container’s styles meet the queried condition, we can apply styles to other elements. This is the sort of “conditional” styling we’ve wanted on the web for a long time: If these styles match over here, then apply these other styles over there.
CSS Container Style Queries are only available as an experimental feature in modern web browsers at the time of this writing, and even then, style queries are only capable of evaluating CSS custom properties (i.e., variables).
Browser SupportThe feature is still considered experimental at the time of this writing and is not supported by any browser, unless enabled through feature flags.
This browser support data is from Caniuse, which has more detail. A number indicates that browser supports the feature at that version and up.
DesktopChromeFirefoxIEEdgeSafari129NoNo126TPMobile / TabletAndroid ChromeAndroid FirefoxAndroidiOS Safari126No12618.0 Registering a Style Container article { container-name: card; }That’s really it! Actually, we don’t even need the container-name property unless we need to target it specifically. Otherwise, we can skip registering a container altogether.
And if you’re wondering why there’s no container-type declaration, that’s because all elements are already considered containers. It’s a lot like how all elements are position: relative by default; there’s no need to declare it. The only reason we would declare a container-type is if we want a CSS Container Size Query instead of a CSS Container Style Query.
So, really, there is no need to register a container style query because all elements are already style containers right out of the box! The only reason we’d declare container-name, then, is simply to help select a specific container by name when writing a style query.
Using a Style Container Query @container style(--bg-color: #000) { p { color: #fff; } }In this example, we’re querying any matching container (because all elements are style containers by default).
Notice how the syntax it’s a lot like a traditional media query? The biggest difference is that we are writing @container instead of @media. The other difference is that we’re calling a style() function that holds the matching style condition. This way, a style query is differentiated from a size query, although there is no corresponding size() function.
In this instance, we’re checking if a certain custom property named --bg-color is set to black (#000). If the variable’s value matches that condition, then we’re setting paragraph (p) text color to white (#fff).
Custom Properties & Variables .card-wrapper { --bg-color: #000; } .card { @container style(--bg-color: #000) { /* Custom CSS */ } } Nesting Style Queries @container style(--featured: true) { article { grid-column: 1 / -1; } @container style(--theme: dark) { article { --bg-color: #000; --text: #fff; } } } SpecificationCSS Container Queries are defined in the CSS Containment Module Level 3 specification, which is currently in Editor’s Draft status at the time of this writing.
Browser SupportBrowser support for CSS Container Size Queries is great. It’s just style queries that are lacking support at the time of this writing.
- Chrome 105 shipped on August 30, 2022, with support.
- Safari 16 shipped on September 12, 2022, with support.
- Firefox 110 shipped on February 14, 2023, with support.
This browser support data is from Caniuse, which has more detail. A number indicates that browser supports the feature at that version and up.
DesktopChromeFirefoxIEEdgeSafari106110No10616.0Mobile / TabletAndroid ChromeAndroid FirefoxAndroidiOS Safari12612712616.0 Demos!Many, many examples on the web demonstrate how container queries work. The following examples are not unique in that regard in that they illustrate the general concept of applying styles when a container element meets a certain condition.
You will find plenty more examples listed in the References at the end of this guide, but check out Ahmad Shadeed’s Container Queries Lab for the most complete set of examples because it also serves as a collection of clever container query use cases.
Card ComponentIn this example, a “card” component changes its layout based on the amount of available space in its container.
CodePen Embed Fallback Call to Action PanelThis example is a lot like those little panels for signing up for an email newsletter. Notice how the layout changes three times according to how much available space is in the container. This is what makes CSS Container Queries so powerful: you can quite literally drop this panel into any project and the layout will respond as it should, as it’s based on the space it is in rather than the size of the browser’s viewport.
CodePen Embed Fallback Stepper ComponentThis component displays a series of “steps” much like a timeline. In wider containers, the stepper displays steps horizontally. But if the container becomes small enough, the stepper shifts things around so that the steps are vertically stacked.
CodePen Embed Fallback Icon ButtonSometimes we like to decorate buttons with an icon to accentuate the button’s label with a little more meaning and context. And sometimes we don’t know just how wide that button will be in any given context, which makes it tough to know when exactly to hide the icon or re-arrange the button’s styles when space becomes limited. In this example, an icon is displayed to the right edge of the button as long as there’s room to fit it beside the button label. If room runs out, the button becomes a square tile that stacks the icons above the label. Notice how the border-radius is set in container query units, 4cqi, which is equal to 4% of the container’s inline-size (i.e. width) and results in rounder edges as the button grows in size.
CodePen Embed Fallback PaginationPagination is a great example of a component that benefits from CSS Container Queries because, depending on the amount of space we have, we can choose to display links to individual pages, or hide them in favor of only two buttons, one to paginate to older content and one to paginate to newer content.
CodePen Embed Fallback Articles & Tutorials General Information Article on Jun 14, 2024 CSS Container Queries container-queries Robin Rendle Article on Jun 14, 2024 CSS Container Queries container-queries Robin Rendle Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Una Kravets Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Mathias Hülsbusch Container Size Query Tutorials Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Jhey Tompkins Article on Jun 14, 2024 CSS Container Queries container-queries Dan Christofi Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier Container Style Queries Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Almanac References Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Article on Jun 14, 2024 CSS Container Queries container-queries Geoff Graham Related Guides Article on Jun 14, 2024 CSS Container Queries container-queries Andrés Galante Article on Jun 14, 2024 CSS Container Queries container-queries Chris Coyier References- Container Queries: A Quick Start Guide (OddBird)
- CSS Containers, What Do They Know? (OddBird)
- A Primer On CSS Container Queries (Smashing Magazine)
- CSS Container Queries: Use-Cases And Migration Strategies (Smashing Magazine)
- An Interactive Guide to CSS Container Queries (and Demos) (Ahmad Shadeed)
- Container queries – Designing in the Browser (web.dev)
- CSS container queries (Mozilla Developer Network)
- Container Queries are going to change how we make layouts (Kevin Powell)
- Container queries and units (Zach Saucier)
- Container Query Units and Fluid Typography (ModernCSS)
CSS Container Queries originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.
Ask LukeW: Dynamic Preview Cards
After adding the Ask Luke feature to my site last year, I began sharing interesting questions people asked and their answers. But doing so manually meant creating an image in Photoshop and attaching it to posts on Twitter, LinkedIn, etc. Now with dynamic Open Graph previews, these preview cards get created on the fly- pretty sweet.
Ask Luke is an AI-powered conversational interface that uses the thousands of articles, videos, audio files, and PDFs I've created over the years to answer people's questions about digital product design. Every time the system answers a question, it does so dynamically. So technically, each answer is unique.
To make each question and answer pair sharable, the first step was to enable creating a unique link to it. The second was to use Vercel's image generation library to create a preview card each time someone makes a link.
The dynamic preview card for each question and answer pair includes as much of the question we can in addition to a bit of the response. It also adapts to varying question and answer lengths since it is generated dynamically.
When shared on Twitter, LinkedIn, Apple Messages, Slack, and any other application that supports Open Graph previews, an image with the question and answer is displayed providing a sense of what the link leads to.
Thanks to Yangguang Li, Thanh Tran, and Sam for the tips and help with this.