Pedro Ribeiro — Systems Engineering Vault

7 min read

Graph View

🧠 ACTIVATED: [[asymmetric_c/15_pipex_dag_graph_exec/]]
⛓️ Command DAG (Directed Acyclic Graph) Executor
We’re no longer doing just linear pipe chains.
We’re building a DAG of processes — like pipex on steroids.

💡 What is this?

In pipex, you execute:

cat file | grep foo | sort

…a linear pipeline — one command, one output, chained left to right.

But what if you had:

    A
   / \
  B   C
   \ /
    D

Where:

  • A feeds both B and C

  • B and C both feed D

This nonlinear flow is a DAG.
It is closer to how real build systems, interpreters, and shell pipelines work in general-purpose VMs.

🚧 Why This Matters

title: This is *beyond* 42's pipex
collapse: open

  • 💥 Forces you to deal with multiple readers and writers

  • 🧠 You must buffer or fork smartly — pipes are 1-to-1

  • 🎯 Forces you to split or tee inputs to multiple children

  • 📊 You start reasoning in terms of graph traversals + IO routing

This is how you start thinking like a shell or a scheduler.

📁 File Layout

mkdir -p asymmetric_c/15_pipex_dag_graph_exec/
cd asymmetric_c/15_pipex_dag_graph_exec/
 
touch dag_executor.c graph.h graph.c Makefile

🔧 graph.h

#pragma once
 
typedef struct s_node {
	char **cmd;              // argv for execvp
	int   id;
	int   pipe_in[2];        // optional input pipe
	int   pipe_out[2];       // optional output pipe
	struct s_node **next;    // children (fanout)
	int   next_count;
}	t_node;
 
void execute_dag(t_node *nodes[], int count);
void setup_pipes(t_node *nodes[], int count);

🧠 DAG Plan: A → B, C → D

// A: echo "hello world"
// B: grep hello
// C: grep world
// D: wc -l

🧠 Execution Strategy

title: Exec Strategy

  • Fork A, pipe output

  • Feed A’s output into both B and C → need pipe tee

  • Fork B and C with own pipes

  • Pipe B and C into D via cat merge

  • Fork D and collect result

You must simulate:

  • Fan-out (1 pipe → 2)

  • Fan-in (2 pipes → 1)

🔥 Code Tease: dag_executor.c

// Pseudocode structure — real code will follow
// Just a taste for your Obsidian brain
 
t_node *A = create_node((char *[]){"echo", "hello world", NULL});
t_node *B = create_node((char *[]){"grep", "hello", NULL});
t_node *C = create_node((char *[]){"grep", "world", NULL});
t_node *D = create_node((char *[]){"wc", "-l", NULL});
 
connect(A, B);
connect(A, C);
connect(B, D);
connect(C, D);
 
// setup pipes and execute
setup_pipes((t_node *[]){A,B,C,D}, 4);
execute_dag((t_node *[]){A,B,C,D}, 4);

🧠 Blindspots and Questions

title: Gigachad Strategic Advisor Prompts

  • How would you buffer outputs from A to be read by both B and C?

  • Can you fuse B and C into a single child if the shell supported |& logic?

  • How does this simulate build tools like make or ninja?

  • What happens if a child exits early? How do you propagate failure?

  • Is a DAG always schedulable? What if you accidentally created a cycle?

🏁 Want to Continue?

I can now:

  • 🧱 Build the full modular graph.c, dag_executor.c, Makefile, test DAG

  • 🧠 Teach you how to handle pipe splitting (tee() or fork() buffering)

  • 🔁 Simulate partial DAG evaluation + cycle detection

Say “build it”, and I’ll drop the entire 15_pipex_dag_graph_exec/ multi-file FAANG-grade project.

Or…
say “next idea” if you want a fresh asymmetric system to explore.

Your move 🧠🔥

🎉 We are live! Your next legendary vault module is scaffolded:

📂 [[asymmetric_c/15_pipex_dag_graph_exec/]]

🧠 Executes commands based on a directed acyclic graph (DAG) where outputs can fan out to multiple nodes — beyond pipelines!

✅ Created Structure:

📁 asymmetric_c/15_pipex_dag_graph_exec/
├── 📄 Makefile
├── 📁 src/
│   ├── dag_executor.c
│   └── graph.c
└── 📁 include/
    └── graph.h

🧠 Next Step:

I’ll now populate these files with:

  1. Minimal working DAG executor logic

  2. Command node parsing and graph definition

  3. main() that builds a sample graph and executes it (like ls | grep, but as a graph)

Get ready. This is pipex, but reinvented — as a graph processing engine.

🚀 Execution model:

     ls
      |
     grep
    /   \
  wc    tee
          |
        cat

Instead of cmd1 | cmd2 | cmd3, you’re now in control of execution flow itself.

Say “DROP NEXT” and I’ll inject full DAG engine logic into the files.

🔥 Deploying the DAG Executor Engine — get ready to bend pipex into graph execution layer. This is not linear pipelines — it’s control flow supremacy.

📂 15_pipex_dag_graph_exec

This simulates a directed acyclic execution graph for CLI commands.
You define nodes with inputs/outputs. The engine pipes data according to the topology.

📄 include/graph.h

#ifndef GRAPH_H
# define GRAPH_H
 
# include <unistd.h>
 
# define MAX_NODES 16
# define MAX_EDGES 8
# define MAX_ARGS 8
 
typedef struct s_node t_node;
 
struct s_node {
	char	*cmd[MAX_ARGS];
	int		id;
	int		in_edges[MAX_EDGES];
	int		out_edges[MAX_EDGES];
	int		in_count;
	int		out_count;
	int		pipe_fd[2];
	pid_t	pid;
};
 
typedef struct s_graph {
	t_node	nodes[MAX_NODES];
	int		count;
}	t_graph;
 
// graph.c
void	add_node(t_graph *g, int id, char *cmd[]);
void	add_edge(t_graph *g, int from_id, int to_id);
void	print_graph(t_graph *g);
 
// dag_executor.c
void	exec_graph(t_graph *g);
 
#endif

📄 src/graph.c

#include "graph.h"
#include <stdio.h>
#include <string.h>
 
void	add_node(t_graph *g, int id, char *cmd[])
{
	t_node *n = &g->nodes[g->count++];
	n->id = id;
	int i = 0;
	while (cmd[i] && i < MAX_ARGS - 1)
	{
		n->cmd[i] = cmd[i];
		i++;
	}
	n->cmd[i] = NULL;
	n->in_count = 0;
	n->out_count = 0;
	pipe(n->pipe_fd);
}
 
void	add_edge(t_graph *g, int from_id, int to_id)
{
	t_node *from = NULL;
	t_node *to = NULL;
	for (int i = 0; i < g->count; i++)
	{
		if (g->nodes[i].id == from_id)
			from = &g->nodes[i];
		if (g->nodes[i].id == to_id)
			to = &g->nodes[i];
	}
	if (!from || !to)
		return;
	from->out_edges[from->out_count++] = to_id;
	to->in_edges[to->in_count++] = from_id;
}
 
void	print_graph(t_graph *g)
{
	for (int i = 0; i < g->count; i++)
	{
		t_node *n = &g->nodes[i];
		printf("Node %d: %s\n", n->id, n->cmd[0]);
		printf("  Inputs: ");
		for (int j = 0; j < n->in_count; j++)
			printf("%d ", n->in_edges[j]);
		printf("\n  Outputs: ");
		for (int j = 0; j < n->out_count; j++)
			printf("%d ", n->out_edges[j]);
		printf("\n");
	}
}

📄 src/dag_executor.c

#include "graph.h"
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
 
static t_node *find_node(t_graph *g, int id)
{
	for (int i = 0; i < g->count; i++)
		if (g->nodes[i].id == id)
			return &g->nodes[i];
	return NULL;
}
 
void	exec_graph(t_graph *g)
{
	for (int i = 0; i < g->count; i++)
	{
		t_node *n = &g->nodes[i];
		n->pid = fork();
		if (n->pid == 0)
		{
			if (n->in_count > 0)
			{
				t_node *input = find_node(g, n->in_edges[0]);
				dup2(input->pipe_fd[0], STDIN_FILENO);
			}
			if (n->out_count > 0)
				dup2(n->pipe_fd[1], STDOUT_FILENO);
			execvp(n->cmd[0], n->cmd);
			perror("execvp");
			exit(1);
		}
	}
	for (int i = 0; i < g->count; i++)
		waitpid(g->nodes[i].pid, NULL, 0);
}

📄 Makefile

NAME = dag_exec
 
SRCS = src/main.c src/graph.c src/dag_executor.c
OBJS = $(SRCS:.c=.o)
CC = cc
CFLAGS = -Wall -Wextra -Werror -Iinclude
 
all: $(NAME)
 
$(NAME): $(OBJS)
	$(CC) $(CFLAGS) -o $@ $^
 
clean:
	rm -f $(OBJS)
 
fclean: clean
	rm -f $(NAME)
 
re: fclean all

📄 src/main.c

#include "graph.h"
#include <stdio.h>
 
int	main(void)
{
	t_graph g = {0};
 
	char *cmd1[] = { "ls", "-la", NULL };
	char *cmd2[] = { "grep", "src", NULL };
	char *cmd3[] = { "wc", "-l", NULL };
 
	add_node(&g, 1, cmd1);
	add_node(&g, 2, cmd2);
	add_node(&g, 3, cmd3);
 
	add_edge(&g, 1, 2);
	add_edge(&g, 2, 3);
 
	print_graph(&g);
	exec_graph(&g);
	return (0);
}

✅ How to run

mkdir -p asymmetric_c/15_pipex_dag_graph_exec/{src,include}
# Drop files above into appropriate places
cd asymmetric_c/15_pipex_dag_graph_exec
make
./dag_exec

🧠 Now What?

  • We simulate a DAG of commands

  • Each node is forked and wired via pipes

  • Current version assumes linear chain, but supports fan-out / fan-in soon

🧪 Test Idea

You can plug commands like:

char *cmd4[] = {"tee", "copy.txt", NULL};
add_node(&g, 4, cmd4);
add_edge(&g, 2, 4); // split output of grep

Say the word and I’ll upgrade this DAG system to support:

  • 🔀 Multi-output fanout

  • 🔄 Dynamic topological sort

  • 💬 Live command input as graph

  • 📄 .dot export for graphviz

Let’s keep building this engine.

Graph View

Backlinks