- name
Use when building your own cognee processing — writing custom tasks, chaining them into a pipeline with run_custom_pipeline or the lightweight run_pipeline (from cognee.pipelines import run_pipeline),
SKILL.md
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add topoteretes/cognee --skill cognee-custom-pipelines -a cursor -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add topoteretes/cognee --skill cognee-custom-pipelines -a claude-code -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add topoteretes/cognee --skill cognee-custom-pipelines -a codex -yPaste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add topoteretes/cognee --skill cognee-custom-pipelines -a grok -ySend the copied message in a Grok Bot chat. The Bot fetches this SKILL.md from GitHub and adds it to the skills all your Bots share.
Install this skill: https://github.com/topoteretes/cognee/tree/HEAD/.agents/skills/cognee-custom-pipelinesFrom the publisher
Everything cognee does runs as a pipeline: an ordered list of tasks,
each a plain Python function whose output feeds the next one. remember()
is the right tool for ordinary ingestion. Build a pipeline when you need
processing cognee does not ship: your own extraction, your own node types,
or a post-processing step over the graph.
import cognee
from cognee.modules.pipelines import Task
from cognee.tasks.storage import add_data_points
from cognee.low_level import DataPoint
class Person(DataPoint):
name: str
metadata: dict = {"index_fields": ["name"], "identity_fields": ["name"]}
async def extract_people(data_items: list) -> list[Person]:
people = []
for item in data_items: # always a list, see below
text = item if isinstance(item, str) else ""
people += [Person(name=n.strip()) for n in text.split(",") if n.strip()]
return people
result = await cognee.run_custom_pipeline(
tasks=[
Task(extract_people, needs_llm=False),
Task(add_data_points, needs_llm=False), # store in graph + vector DBs
],
data=["Ada Lovelace, Alan Turing"],
dataset="people",
)
There are three, and two share the name run_pipeline:
cognee.run_custom_pipeline(tasks, data=None, dataset="main_dataset", user=None, incremental_loading=False, data_per_batch=20, run_in_background=False, pipeline_name="custom_pipeline", data_cache=False, ...) returns {dataset_id: PipelineRunInfo} (the started run when
run_in_background=True). With data=None it runs over the dataset's
existing documents (Data rows).
from cognee.modules.pipelines import Task
from cognee.modules.pipelines.models import PipelineContext
from cognee.modules.pipelines.tasks.task import task_summary
from cognee.pipelines import Drop
@task_summary("Tagged {n} chunk(s)")
async def tag_chunks(chunks: list, ctx: PipelineContext = None, label: str = "x"):
for chunk in chunks:
chunk.metadata["label"] = label
return chunks # or yield per item; return/yield Drop to discard
tag = Task(tag_chunks, label="reviewed", batch_size=10, needs_llm=False)
async def, a generator, an async generator, or a plain
def. Extra Task(fn, *args, **kwargs) arguments are passed after the
pipeline data.needs_llm=False on tasks that never call an LLM lets an LLM-free
pipeline skip the LLM connection check.ctx (injected by the parameter name ctx) carries user,
data_item, dataset, pipeline_run_id, pipeline_name, and extras.task.with_config(batch_size=..., **kwargs) returns a modified copy.run_custom_pipeline
or the orchestrator, and the first task receives it as a one-element
list ([data_item]), not the bare item. The lightweight run_pipeline
passes data to the first task unchanged.data_per_batch (default 20) is how many documents run at the same
time. It is a concurrency limit, not a batch size.batch_size belongs to the consumer. A task's batch_size decides how
the previous task's generator output is grouped before it is passed in.
Generator tasks always hand over lists; a coroutine or function hands over
its single return value.enriches=True: if the task returns None, its input is passed on
unchanged (coroutines and functions only, not generators).Drop: returning or yielding it removes that item from the stream.DataPoint passing through is stamped automatically with where it
came from (source_pipeline, source_task, source_user, …).add_data_points(data_points, custom_edges=None, graph_only=False) writes a list of DataPoints to the graph and indexes
their index_fields in the vector DB. It returns the same list, so it can
sit mid-chain. Give every node type identity_fields so repeated runs merge
instead of duplicating (see the cognee-custom-graph-models skill).
cognee.memify, cognee-cli memify, POST /api/v1/memify and improve()'s
extraction_tasks / enrichment_tasks are deprecated and removed in the next
release. Run a post-processing step over the
graph with run_custom_pipeline: the task reads the graph itself, and one
placeholder item as data makes the chain run once instead of once per
document.
from cognee.infrastructure.databases.graph import get_graph_engine
async def enrich_graph(_items: list):
graph = await get_graph_engine() # the dataset's graph inside the pipeline
nodes, edges = await graph.get_graph_data()
return [...] # new DataPoints for add_data_points
await cognee.run_custom_pipeline(
tasks=[Task(enrich_graph, needs_llm=False), Task(add_data_points, needs_llm=False)],
data=[{}],
dataset="people",
)
For cognee's own enrichment (session bridging, feedback weights, conflict
review, global context index) call improve().
status = await cognee.datasets.get_status([dataset_id], pipeline_names=["custom_pipeline"])
Without pipeline_names it reports only cognify_pipeline. It returns
{str(dataset_id): PipelineRunStatus} ({str(dataset_id): {pipeline_name: status}} for several pipeline_names): DATASET_PROCESSING_STARTED,
_COMPLETED, or _ERRORED (_INITIATED exists only on legacy rows). The value run_custom_pipeline
returns per dataset is a PipelineRunInfo instead, whose class names the
outcome: PipelineRunCompleted, PipelineRunAlreadyCompleted,
PipelineRunErrored, and so on.
run_pipeline. The one imported via
from cognee.pipelines import run_pipeline wants task()
specs called (extract(), not extract) and raises TypeError
otherwise; the orchestrator in cognee.modules.pipelines wants Task
objects and raises WrongTaskTypeError otherwise.run_custom_pipeline accepts only Task objects despite its type hint;
string task names are not resolved.Task (ValueError: Unsupported task type): bound methods and functools.partials of a plain (non-generator)
sync function, and callable objects (instances with __call__). Generator
and async variants, plain functions, and lambdas work. When in doubt, wrap it
in a plain def / async def.run_custom_pipeline does not run database migrations. On an existing
database, run await cognee.run_migrations() (or any remember() first).identity_fields (or Dedup() fields). examples/guides/custom_data_models.py
and examples/guides/custom_tasks_and_pipelines.py have this bug; don't
copy it.run_custom_pipeline → orchestrator run_pipeline (checks write
permission, takes the per-dataset lock, records a PipelineRun) →
run_tasks (a semaphore of data_per_batch, one chain per document) →
run_tasks_base (streams each task's output into the next, batching by the
consumer's batch_size, injecting ctx, stamping provenance).
cognee/modules/pipelines/__init__.pyTask, task(), TaskSpec, BoundTask, @task_summary: cognee/modules/pipelines/tasks/task.pycognee/modules/pipelines/operations/pipeline.py; execution:
run_tasks.py, run_tasks_base.py, run_tasks_data_item.pycognee/modules/pipelines/operations/run_pipeline.py,
exported from cognee/pipelines/cognee/modules/pipelines/models/PipelineContext.pyrun_custom_pipeline: cognee/modules/run_custom_pipeline/run_custom_pipeline.pycognee/tasks/storage/add_data_points.pycognee/tasks/README.mdExamples:
examples/demos/custom_pipelines/custom_pipeline_single_object_example.py:
the best reference. It runs over added documents, does LLM extraction into
typed DataPoints, then recalls. Its models declare identity with Dedup()
(the Annotated alternative to metadata["identity_fields"]).examples/demos/custom_pipelines/organizational_hierarchy/: low-level
run_tasks, no LLM, dedup via identity_fields, status polling.examples/demos/custom_pipelines/custom_cognify_pipeline_example.py:
rebuilds add + cognify from the default task list.cognee/tasks/ subpackage for its
stage, export it from that package's __init__.py, follow the template in
cognee/tasks/README.md, and add a unit test under cognee/tests/unit/tasks/.cognee/tests/unit/modules/pipelines/ (runner semantics,
context, provenance, rollback) and cognee/tests/unit/pipelines/ (the
lightweight API).© 2026 Skills Rack.
---name: cognee-custom-pipelinesdescription: Use when building your own cognee processing — writing custom tasks, chaining them into a pipeline with…---# .agents/skills/cognee-custom-pipelines/SKILL.mdOverview
Use when building your own cognee processing — writing custom tasks, chaining them into a pipeline with run_custom_pipeline or the lightweight run_pipeline (from cognee.pipelines import run_pipeline), storing custom DataPoints with add_data_points, running custom enrichment over the existing graph, checking pipeline run status, or debugging how data flows between tasks (batch_size, data_per_batch, ctx, Drop, enriches).
FAQ
Download .agents/skills/cognee-custom-pipelines/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
The publisher keeps it at https://github.com/topoteretes/cognee.
Claude, Cursor, Codex, and Grok Build can read a SKILL.md you add from the publisher's repository. Grok Bot installs it when you send the skill's GitHub folder link in a chat.