Estimated reading time at 200 wpm: 9 minutes
The Slop Sieve is an Python application that was developed as a defensive linguistic tool for writers like me, operating in an era of heightened AI surveillance. As algorithmic “detection” becomes more prevalent, writers who produce precise or rhythmic prose are increasingly at risk of being flagged as non-human. This project history documents the rationale, technical architecture, and implementation of a custom “internal radar” designed to identify these risk factors before publication.
Whether or not you agree our Fat Disclaimer applies
1. Rationale: The Need for an Internal Radar
It is important to clarify that the Slop Sieve is not an attempt to “humanise” AI-collaborative work. In the current literary climate, there is an increasing obsession with detecting AI involvement, often leading to “suspect” flags even on purely human-generated content. Because 100% human authorship can no longer be guaranteed to pass these arbitrary “detection” hurdles, the goal shifted from trying to achieve an impossible standard of “purity” to simply understanding and manoeuvring inside linguistic territories.
The primary goal is to acknowledge that Large Language Models (LLMs) leave distinct statistical footprints. These include overused “shimmering” vocabulary, specific structural transitions (internal summaries), and a lot of sentence length variance (the “robotic drone”). For a writer, the Sieve serves as a diagnostic tool. By identifying where a manuscript aligns with these machine-generated probability curves, the writer can navigate a world where they might otherwise be unfairly targeted by “AI hunters.”
2. Python Environment and Project Structure
The project required a robust and isolated environment to handle complex linguistic libraries without interfering with the underlying operating system.
Verification and Version Control
Before development began, it was necessary to verify that the system was running a modern version of Python (3.10 or higher) capable of supporting the latest NLP models. This was checked via the terminal:
python3 --versionCreating the Application Space
Rather than letting the project sit in a cluttered downloads or home folder, a dedicated directory was established within an Applications folder. This mimics a professional software installation structure:
mkdir -p ~/Applications/slop-sieve
cd ~/Applications/slop-sieveThe Virtual Environment (.venv)
To ensure that libraries like spaCy and pandas did not conflict with system-level software, a Virtual Environment was created. This acts as a private “sandbox” for the project. By using the -m venv command, I generated a local Python executable and a site-packages folder dedicated solely to the Slop Sieve.
python3 -m venv .venv
source .venv/bin/activateThis isolation is a critical step for a “noob” developer; it means if a library installation goes wrong, the damage is contained within the project folder and cannot break the computer’s core functions.
3. Technical Architecture: Why spaCy?
When building a tool to detect “slop,” a basic word-search or “string check” is insufficient. A simple search for “realise” would miss “realised,” “realising,” or “realises.” More importantly, it wouldn’t understand the context of the word—whether it is being used as a verb, a noun, or part of a specific grammatical structure.
Community and Open Source
I chose spaCy, a professional-grade Natural Language Processing (NLP) library that emerged from the open-source community. Originally developed by Matthew Honnibal and Ines Montani (Explosion AI), spaCy was built with an “industrial-strength” philosophy. It was designed specifically to be fast and useful in real-world production, rather than just academic research.
Crucially, spaCy is free and open-source software. It thrives on a massive global community of contributors who keep its linguistic models up to date. By choosing spaCy, the Slop Sieve taps into years of collective development, allowing a single writer to use the same linguistic intelligence used by major technology companies to parse and understand human language.
Syntactic Intelligence
Unlike a basic search, spaCy provides a “syntactic understanding” of the document. It performs:
- Tokenisation: Breaking the text into meaningful pieces (words, punctuation).
- Part-of-Speech (POS) Tagging: Identifying if a word is a noun, verb, adjective, etc., based on its usage in the sentence.
- Lemmatisation: Reducing every word to its dictionary root.
By using spaCy, the Sieve becomes “literate.” It doesn’t just see characters; it sees the mechanics of the sentence.
The Grammatical “Matcher”
To find complex patterns like “Internal Summaries” (where a character’s thoughts are told rather than shown), I utilised the spaCy Matcher.
# A snippet of the logic used to find internal summaries
matcher.add("INTERNAL_SUMMARY", [[
{"POS": {"IN": ["PRON", "PROPN"]}},
{"POS": {"IN": ["VERB", "ADP", "DET", "NOUN", "CCONJ"]}, "OP": "*", "LENGTH": {"<=": 4}},
{"LEMMA": {"IN": ["feel", "experience", "notice", "sense", "realize", "understand"]}}
]])How it works:
- POS (Part of Speech): The script looks for a Pronoun (He/She) or Proper Noun (a character’s name).
- OP (Operator): It allows for a few optional short words in between, ensuring the pattern is flexible enough to catch variations like “He then suddenly realised.”
- LEMMA: It searches for the “root” of the verb. This ensures that “He realised,” “He was realising,” and “He slowly realises” are all caught by a single rule. Note that while the code uses American-style lemmas like “realize” for the internal dictionary, the input text is treated with UK English standards.
Lemmatisation: Catching the Root
To identify “AI-isms” (words frequently overused by LLMs), the script uses “Lemmatisation.”
# Identifying overused terms regardless of tense
ai_isms_roots = ["delve", "tapestry", "shimmering", "resonate"]
for token in doc:
if token.lemma_.lower() in ai_isms_roots:
# This flags "shimmering", "shimmered", and "shimmers" equally.Without spaCy’s lemmatisation, I would have to provide a list of every possible variation of every word. With spaCy, I only need the “root,” and the engine handles the rest.
4. Rhythm Analysis: The Robotic Drone Check
AI text often suffers from a lack of rhythmic variance. It tends to produce sentences of similar lengths, creating a metronomic effect. The Slop Sieve uses standard deviation to calculate this “flatness.”
# Mathematical rhythm check
import numpy as np
# Measure every sentence length in a paragraph
sent_lens = [len(sent) for sent in doc.sents]
# Calculate the 'spread' of those lengths
variance = np.std(sent_lens)
if variance < 4.0:
# Flag as a 'Robotic Drone' (too flat)The Logic:
Human writing usually features a mix of short, punchy fragments and longer, flowing clauses. If the variance is too low (below 4.0), it suggests a machine-like consistency. Interestingly, a human writer might intentionally use a flat rhythm to convey stillness or gravity, yet the “AI hunter” logic would still flag this as a machine trait.
5. Troubleshooting and Iteration
The development process was not without hurdles. Initial versions of the script suffered from NameError issues where core functions were referenced before they were defined. This was resolved by unifying the linguistic engine and the GUI logic into a single, cohesive script.
Furthermore, I found that the script needed to be explicit about UK English standards. While the underlying NLP models often default to American English, the Sieve was adjusted to ensure that “realised” and “realised” are treated as functionally identical to the logic, respecting the writer’s regional grammar.
6. Environment: Developing for BigLinux
The application was developed specifically to run on BigLinux, a distribution known for its focus on usability and its use of the KDE Plasma desktop environment. This choice of operating system heavily influenced the development of the user interface and system integration.
Native File Picking with kdialog
Standard Python file windows (via Tkinter) often feel “foreign” on a Linux desktop; they ignore system bookmarks, network places, and “Recent Files” sidebars. To ensure a seamless experience, the script was programmed to bypass the standard Python window and call kdialog, the native file manager for BigLinux/KDE.
# Calling the Linux system file picker
import subprocess
cmd = ["kdialog", "--getopenfilename", "/home/wwww/"]
file_path = subprocess.check_output(cmd).decode("utf-8").strip()This bridge allows the writer to navigate their manuscript folders using the exact same interface they use for all other tasks on BigLinux.
7. Deployment: The Desktop Launcher
The final stage involved making the tool accessible via the system’s application menu. This required creating a .desktop configuration file.
Virtual Environment Integration
The application lives in a specific directory: /home/wwww/xxxxxx/yyyyy/zzzz/. Because it requires specific libraries, it runs inside a Virtual Environment (.venv). This is an isolated “container” that holds the exact version of Python and the linguistic models needed.
The Launcher Shortcut
A launcher file was created to point the system toward this container. Initial attempts to launch via a simple path failed in the GUI environment; the solution was to wrap the command in a bash shell to ensure the environment was correctly initialised.
[Desktop Entry]
Version=1.0
Type=Application
Name=Slop Sieve
Exec=bash -c "/home/wwww/xxxx/yyyy/.venv/bin/python /home/wwww/xxxx/yyyy/slop_sieve_gui.py"
Terminal=false
StartupNotify=true
Icon=document-editIn BigLinux, after creating this file, a system command (kbuildsycoca5 or kbuildsycoca6) is often required to refresh the menu database so the new “Sieve” app appears in the start menu search results.
Conclusion
The Slop Sieve represents a specialised adaptation to the current state of digital publishing. It is essential to reiterate that this is not an attempt to “humanise” AI-collaborative work or to deceive the reader. Instead, it is a response to an increasingly artificial situation where even purely human prose can be intercepted and interrogated by “AI hunters.” There is no way for an author to prove that they ‘didn’t use AI assistance’. This means that if a bunch of reviewers come to beliefs based on AI analysis, that a piece of work involved AI – there is no ‘defence’. It’s like how does a person prove that they did not have sexual relations with ‘that woman’, if the world thinks otherwise.
In this climate, being a disciplined or precise writer can be a liability. The Sieve does not fix the writing; it reveals the mathematical “fingerprints” that might be misinterpreted as machine-generated. By combining linguistic modelling, statistical rhythm analysis, and native BigLinux integration, it provides a streamlined way for writers to audit their work. It is a tool for navigation, helping authors maintain their visibility in a world where the boundary between human and machine logic is being aggressively—and often inaccurately—policed.
