kovyrin

6 ms·

1.
A Reliable Workflow for Building with AI Agents Using PRDs(kovyrin.net)
2 points by kovyrin 1y ago|1 comments
2.
by kovyrin 1y ago
I wrote about a workflow I’ve been using to manage AI agents across multiple projects using structured PRDs and task lists. It’s a manual, markdown-based approach that ended up being more reliable than multi-agent automation tools for real-
3.
by kovyrin 5y ago
> I think it can be summarized as "recklessness" (i.e. jumping the gun without proper understanding of/regard to foundational concepts) Well… I tried to approach it properly. You google or look up in wikipedia a bunch of i
4.
by kovyrin 12y ago
Great comment! Thanks a lot for sharing this.
5.
by kovyrin 12y ago
Those weren't outages for a specific instance. But from the whole pool of instances we were running 2-3 would have networking issues, random unexplained hangs requiring an instance restart, huge CPU performance drops, IO hangs, etc, et
6.
by kovyrin 12y ago
We had reserved instanced and regular ones, we did not see any patterns in stability issues between those. Re: instance types - I do not really remember which instances we were using to be honest. And as for the scaling up and down - we had
7.
by kovyrin 12y ago
Honestly, just as it is with many SAAS companies at some scale, we do not need care about any specific customer's traffic anymore. Simply because we get so much traffic already from our existing customers, that none of the new customer
8.
by kovyrin 12y ago
We get free inter-DC connectivity for free with Softlayer. There really is no difference for us between connecting to a server within a DC or between datacenters (aside from added latency, of cource) – the same private address space, the sa
9.
by kovyrin 12y ago
No, I really do not think going to EC2 could be beneficial in any way in terms of improving resiliency compared to Softlayer. SL allows you to control which VLANs your box will end up on. VLANs could be treated as racks (since they do not a
10.
by kovyrin 12y ago
AFAIK Softlayer has hourly-based rental for real hardware as well. Never tried that, so could not comment on it though.
11.
by kovyrin 12y ago
Softlayer manages the network for us. They offer a fully isolated backend network (frontend connection to the world is optional). For frontend connections we simply use iptables.
12.
by kovyrin 12y ago
My personal experience within the last 7 years: * Softlayer – the best option in terms of quality of service, quality of hardware, quality and size of the infrastructure (geo distribution, etc). * Rackspace – nice, until you grow enough to
13.
by kovyrin 12y ago
With Softlayer I get multiple locations (transparently routed 100% transparent backend connectivity), get 1-2 hrs provisioning speed, I do not need to worry about networking and hardware. Failed drives (when they fail, which happens rarely
14.
by kovyrin 12y ago
Colo - that's an option I'll try to stay away from as long as it is humanly possible. All of my experiences with colo hardware caused a lifetime of pain so that I'm happy to be paying SL a premium for their outstanding servic
15.
by kovyrin 12y ago
Yeah, that (worker nodes, async processing, etc) seems like one of those ideal use-case for clouds that I could definitely support as a viable option for companies that aren't ready for all-in cloud deployment.
16.
by kovyrin 12y ago
1-2 hours is their new server provisioning time. For HW issues we use nagios (that checks raid health and ECC memory health regularly) and at the moment we just file a ticket with SL about the issue showing them the output from our monitori
17.
by kovyrin 12y ago
Yes, it really does look like @stephen-mw is talking about a migration to an owned/colocated hardware. And that's a very different kind of hairy mess I'd prefer to stay away from as long as humanly possible.
18.
by kovyrin 12y ago
This comment is pure gold! That's exactly what I wanted to explain here and you did it so well. Thanks!
19.
by kovyrin 12y ago
Absolutely, 100% agree. Thanks for the comment!
20.
by kovyrin 12y ago
If only it were that simple... But every second relatively young engineer I interview points out that "But Netflix! Look how good the cloud works for them!" and once again I need to explain, that Netflix spends millions in enginee
21.
by kovyrin 12y ago
Oh, there wasn't a handful of servers after we finished the migration (we have migrated a bit late IMO, so we had a lot of traffic even back then). And today, with much larger infrastructure, with hardware clusters specifically tailore
22.
by kovyrin 12y ago
Yep, that is why I specifically noted in the article, that given enough resources, it is possible to survive and even thrive in the cloud (Netflix being one of the best examples of that), but in case of a startup it is not aways the best id
23.
by kovyrin 12y ago
Yep, that is why in the article I specifically pointed out that we have migrated from EC2, but we are still a loyal customer of some of AWS services and those work really nice for us.
24.
by kovyrin 12y ago
We use DNS-based load balancing along with an HA pair of load balancers in each datacenter. If the DNS health check fails, we stop sending traffic to a failing frontend LB. If failing LB is dead, we move its IP to the other one. DNS TTL is
25.
by kovyrin 12y ago
Yes, completely agree. Maybe I'd just add one more case here: some users are OK with locking themselves up to AWS by treating it as a platform from the day one and building on top of AWS database/queue/etc services. For those
26.
by kovyrin 12y ago
In my personal opinion (based on some real experience) EC2 for a startup makes a lot of sense for prototyping your application and your infrastructure (when you don't really know what is it you're building and what components you&
27.
by kovyrin 12y ago
Oh, I know what you're talking about. We too had some instances (actually, a lot of those) that would run for a year with no issues. The problems started around the time you tried to push EC2 instances beyond an "idle, handling so
28.
by kovyrin 12y ago
We are still a loyal customer for some of their services. For example, we still use S3 for off-site backups and Route53 is still our primary DNS provider. For load balancing we have moved to a Route53 (health checks and round-robin) + a gro
29.
by kovyrin 12y ago
Actually, it depends. Stability and performance wise, I'd say our Lucene-based search layer has seen the most impressive jump. But yeah, Mongo instances loved the new fast IO as well :-)
30.
by kovyrin 12y ago
Author here. Happy to answer any questions.

More ›