# Log 2023/08/2023-08-11 ## Digitizing note cards I've been schlepping a set of note cards that were left to digitize since 2022.08.28, so pretty much from one year ago, and it's now finally time to get them into the vault. - [[ZK - 2a9 - p - Categorical systems theory is the study and analysis of systems using category theory]] Some of these notes are really old and I think were really from the beginning of me getting a bit more into [[Category Theory]], but it's now so far in the past that I can't really tell what my understanding of the topic at the time was. That card above sounds more like a definition than an actual ZK entry, and it makes me question how much I knew, was it when I was really just discovering things like "[[Category Theory]] is the algebra of composing things"? - [[ZK - 2a5e7 - p - Use prototypes as katas to explore how to implement different patterns and asbtractions using programming languages and concrete technologies]] ## Writing Writing up on the CLI helper commands with geppetto: - [[DRAFT - 2023-08-11 - Building LLM-powered CLI helpers with pinocchio]] ## Kitbashing By linking the previous note (2a5e7) I stumbled across the concept of [[Program Kitbashing]] which I had started to work out during [[Recurse Center]], and I realize how much that is what I have been going at with [[GO GO GOLEMS]], and it's such a fitting name and concept. [[End-User Kitbashing]] is related to. Then by looking at the [[2022-07-22]] page from which the first [[ZK - 2a9 - p - Categorical systems theory is the study and analysis of systems using category theory]] came from, I stumbled across: - [Malleable software](https://notes.andymatuschak.org/zGL5Kx3PXVwb6b8h2xUKqNd) This link really unlocks a lot of further resources that I'm quite excited to be reading up on. ## Connect to AWS Today I've been mostly working on getting sqleton to use the 2 profile config file to deploy as a single container. However, I need to create the dbt profiles file securely, by reading the parameters out of aws SSM. I wanted to check if the container itself has access to SSM, and in order to do that wanted to connect to the running containers. I decided to use my newly written `llmaws` tool to get the commands to do that. ```bash ❯ cat ~/code/wesen/wesen-misc/bash/connect-ecs.sh #!/usr/bin/env bash # Use fzf to select cluster, task, and container cluster=$(aws ecs list-clusters --query 'clusterArns[]' --output json | jq -r '.'[] | gum choose) task=$(aws ecs list-tasks --cluster $cluster --query 'taskArns[]' --output json | jq -r '.'[] | gum choose) container=$(aws ecs describe-tasks --cluster $cluster --tasks $task --query 'tasks[].containers[].name' --output json | jq -r '.'[] | gum choose) # Connect to the selected container echo "Running aws ecs execute-command --cluster $cluster --task $task --container $container --command "/bin/bash" --interactive" aws ecs execute-command --cluster $cluster --task $task --container $container --command "/bin/bash" --interactive ``` and ```bash #!/bin/bash # Get the list of ECS clusters clusters=$(aws ecs list-clusters --query 'clusterArns[]' --output json | jq -r '.'[]) for cluster in $clusters do echo "Cluster: $cluster" # Get the list of tasks for each cluster tasks=$(aws ecs list-tasks --cluster $cluster --query 'taskArns[]' --output json | jq -r '.'[]) for task in $tasks do echo " Task: $task" # Get the list of running containers for each task containers=$(aws ecs describe-tasks --cluster $cluster --tasks $task --query 'tasks[].containers[].name' --output json | jq -r '.'[]) for container in $containers do echo " Container: $container" done done done ``` However, after that I still get: ``` An error occurred (TargetNotConnectedException) when calling the ExecuteCommand operation: The execute command failed due to an internal error. Try again later. ``` That's when I came across, which apparently is great. - https://github.com/aws-containers/amazon-ecs-exec-checker I first had to enable the execution command capability in the terraform for the service. And indeed the checker tells us: ``` ------------------------------------------------------------- Checks on ECS task and other resources ------------------------------------------------------------- Region : us-east-1 Cluster: arn:aws:ecs:us-east-1:016596391852:cluster/sqleton-ttc-analytics-cluster Task : arn:aws:ecs:us-east-1:016596391852:task/sqleton-ttc-analytics-cluster/12f0aa5c639e4956a5375311f02ccb24 ------------------------------------------------------------- Cluster Configuration | Audit Logging Not Configured Can I ExecuteCommand? | arn:aws:iam::016596391852:user/manuel ecs:ExecuteCommand: allowed ssm:StartSession denied?: allowed Task Status | RUNNING Launch Type | Fargate Platform Version | 1.4.0 Exec Enabled for Task | NO Container-Level Checks | ---------- Managed Agent Status - SKIPPED ---------- ---------- Init Process Enabled (sqleton-ttc-analytics-family:5) ---------- 1. Disabled - "sqleton-ttc-analytics-container" ---------- Read-Only Root Filesystem (sqleton-ttc-analytics-family:5) ---------- 1. Disabled - "sqleton-ttc-analytics-container" Task Role Permissions | arn:aws:iam::016596391852:role/sqleton-task-role-sqleton-ttc-analytics ssmmessages:CreateControlChannel: implicitDeny ssmmessages:CreateDataChannel: implicitDeny ssmmessages:OpenControlChannel: implicitDeny ssmmessages:OpenDataChannel: implicitDeny VPC Endpoints | Found existing endpoints for vpc-1d642c67: - com.amazonaws.us-east-1.s3 SSM PrivateLink "com.amazonaws.us-east-1.ssmmessages" not found. You must ensure your task has proper outbound internet connectivity. Environment Variables | (sqleton-ttc-analytics-family:5) 1. container "sqleton-ttc-analytics-container" - AWS_ACCESS_KEY: not defined - AWS_ACCESS_KEY_ID: not defined - AWS_SECRET_ACCESS_KEY: not defined ``` Which is weird because I set the enable_execute_command: > > To enable execute command for an ECS task in Terraform, you need to set the `enable_execute_command` parameter to `true` in the `aws_ecs_service` resource. Here is an example: > > ```hcl > resource "aws_ecs_service" "service" { > name = "service-name" > cluster = aws_ecs_cluster.cluster.id > task_definition = aws_ecs_task_definition.task.arn > desired_count = 1 > launch_type = "FARGATE" > > network_configuration { > subnets = ["subnet-abcde012", "subnet-bcde012a"] > assign_public_ip = false > } > > enable_execute_command = true > } > ``` > > This will enable the ECS Exec feature for your ECS service. Please note that the ECS Exec feature is only available for tasks that are part of an ECS service. Also, the ECS task definition must be using platform version 1.4.0 or later for Fargate tasks, or the LATEST platform version for EC2 tasks.% So maybe I have to restart the task or so? According to https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html#ecs-exec-enabling-and-using I need to configure the controlchannel settings. After giving it access to the ssm session manager by adding a VPC endpoint and by giving it the proper roles, and restarting the task, I was able to connect to it, and realize that I don't have access to SSM. ```hcl # This is necessary to give us access to the session manager for executing a command on # task containers. resource "aws_vpc_endpoint" "ssm_endpoint" { vpc_id = var.vpc_id service_name = "com.amazonaws.us-east-1.ssmmessages" vpc_endpoint_type = "Interface" subnet_ids = var.public_subnets security_group_ids = [aws_security_group.ecs_security_group.id] } ``` and ``` ```