xavian

4 ms·

1.
by xavian 11y ago
Many many years I was this annoying jr team member that came and asked other people tons of questions. In my case, the other members of the team has something like 10 - 15 years more experience than I. Ultimately what worked was it was re
2.
by xavian 13y ago
I maintain web crawlers for a large internet portal and this stuff is a pain enough to deal with when people unintentionally make their websites recursive, much less INTENTIONALLY. facepalm At least google crawler had issues as well. Than
3.
by xavian 15y ago
The needs of our customer are too transient for reserved yet, so it's primarily been as needed (on demand?) We run into problems during provisioning sometimes and Amazon has to then raise our quota. Some customer architecture and capacity r
4.
by xavian 15y ago
Most of our instances we keep up for months at a time, although they can turn flaky. We tend to use EBS, so we reprovision instances when needed. The biggest overall issue we have with EC2 is clock skew issues that can fool our fault-tolera
5.
by xavian 15y ago
It fluctuates, but most recently somewhere between 34 - 56 instances for dev, staging and production search, crawler and content build clusters. (we index terabytes of data and hundred of millions of documents)
6.
by xavian 15y ago
In the immortal words of Cave Johnson, "Why not?!" :-) Dealing with structured data is easy compared to unstructured data, but there is a vast wealth of unstructured content just waiting to be tapped. I suspect there is so much interest BEC