Over the past months, we’ve constructed a data analysis platform to review long-form text, which uses a number of models, APIs, and custom code to provide feedback to authors. Due to the size of some of our author’s initial drafts, the works range from tens of thousands to hundreds of thousands of words. One of the inherent challenges in the automated summarization of long-form documents is the limitation of “context windows.” In simpler terms, when a machine reads a story, it doesn’t grasp the entire narrative in one go, as humans do. Instead, it reads in chunks or “windows.”
Imagine reading a novel but only looking at ten pages at a time and never revisiting the previous pages. You might understand those ten pages well, but you’d lose the broader context of the story. That’s essentially the challenge many of these machines face. This context window limitation means that the tool might miss broader themes, recurring motifs, or extended character arcs that span the entirety of a long document.
Lately, we’ve been playing around with two popular, tried and true Natural Language Processing tools Sumy and Bert. Both analyze text from different perspectives, providing interesting insights into longer texts. From the view of a classic fairy tale, let’s look at our summary storytellers:
Sumy, The Weaver
Sumy reads a story and imagines it as a grand tapestry. Each sentence is a thread, some intertwined, some standing alone. At times, she plucks the most vibrant threads, weaving them into a miniature tapestry that tries to capture the beauty of the original.
Her Gift? She gave listeners a patchwork summary, reflecting various elements and emotions of the story.
Her Challenge: Sometimes, in trying to capture everything, she might miss the core heartbeat of the tale.
BERT, The Essence Seeker
BERT, on the other hand, is known as the “Essence Seeker.” He often takes a dive deep into the heart of sentences and uses a clustering approach, seeking the core spirit. He’d then surface with a handful of sentences that, to him, felt like the very soul of the story.
His Gift: BERT’s tales were concise, capturing pivotal moments or overarching themes.
His Challenge: In seeking the essence, he sometimes missed the gentle whispers and subtle hues of the story.
Trade-offs
As the boundaries between human creativity and technological prowess blur, tools like Sumy and BERT and others can beckon authors into a realm of narrative understanding. Their unique gifts and challenges remind us of a simple truth: Stories are vast and rich, and while one might capture its hues, another might capture its heart.
And feedback is a gift, no matter the source. It’s what you do with it that matters.
Everyone has a story to tell,
Second Act Fables Team