7 ms·
I'm glad this is out, i'm going to deploy this locally and learn as much about it as possible. Oxide is pretty much the company I dream to work at, both for the
by milon 3y ago
I'm glad this is out, i'm going to deploy this locally and learn as much about it as possible. Oxide is pretty much the company I dream to work at, both for the tech stack, plus the people working there. Thank you Oxide team!
- refulgentis 3y agoCan you get me excited? I spent 20 seconds browsing the homepage and walked away with "so the idea is vertical integration for on-premise server purchases? On custom OS? Why? Why would people pay a premium?" But immediately got myself to "what does a server OS do anyway, doesn't it just launch VMs? You don't need Linux, just the ability to launch Linux VMs" Tell me more? :)
- deleted 3y ago[deleted]
- throwup238 3y agoThe best elevator pitch I've heard is "AWS APIs for on-prem datacenters". They make turn-key managed racks that behave just like a commercial cloud would with all the APIs for VM, storage, and network provisioning and integration you'd expect from AWS, except made to deploy in your company's datacenter under your control.
- kortilla 3y agoThat’s the elevator pitch for open stack
- steveklabnik 3y agoYou are not wrong that OpenStack is sort of similar in a sense, but the difference is that Oxide is a hardware + software product, and OpenStack is purely software.
- capitol_ 3y agoThat just sounds like a bunch of api's on top of linux.
- throwup238 3y agoJust like Dropbox is a bunch of APIs on top of FTP.
- magnawave 3y agoI guess the wildcard is price. AWS's pricing model works kinda at their OMG eyewatering scale - aka all the custom hardware they design is highly cost optimized, but just doing custom hardware has a notable cost. This is easily covered by their scale, to make for their famous margins. [during their low scale times, they did use a good bit of HP/Dell, etc] Oxide seems to be no different (super custom hardware) only major difference being the "in your datacenter" part. Since you own the cost of your datacenter, Oxide has to come in a lot cheaper to even compete with AWS, but how do you do that with low volume [and from the look of it not-cost optimized, but instead fairly tank-like] bespoke hardware? Feels like the pricing / customer fundamentals are going to be pretty rough here outside perhaps a few verticals.
- sarlalian 3y agoDatacenter costs are weird. The first big cost is having a datacenter. However once you have the space, power, cooling and that part makes sense, then the actual hardware going into it can have a pretty decent premium and still be highly competitive with AWS. It will also depend heavily on what you are doing and producing, if the answer to that is a large amount of data, and it needs to transit out of AWS, suddenly the cost of a pretty large datacenter is really cheap in comparison. AWS egress fees have a markup that will make your accountants panic. From a hardware standpoint, once you need GPU compute or large amounts of RAM, the prices get pretty dumb as well.
- kaliszad 3y agoOxide seems to be a lot more efficient than a rack full of 1U servers with each having 2 PSUs + 2 ToR switches + 1 management switch somewhere for all the OOBMs. All those little fans and power conversions eat a lot of power, the fans and the PSUs all cost something too. Also, have fun managing all of that in a secure manner or debugging anything at all. Once you add the VMware licensing you might end up with more or less the same cost up front and quite likely higher overall cost. And I am not even beginning to talk about racking/ stacking of the whole rack. I haven't seen much support even when Dell/EMC owned VMware and together produced the VXRail lineup and the company I used to work for was presented as the reference project in Saxony, Germany at that time. All of the boxes would add up to about 2 standard racks but it was representative of the other bigish customers in that area and time. I imagine, some of the customers will order 1-2 racks half full and over a few years possibly add a few sleds, these will probably demand great GUI/ manual experience and possibly competitive Oracle/ SAP/ MSSQL benchmarks and I can imagine Veeam integreation. Other customers such as the DoE or some big enterprise customers will order whole rows of racks and demand perfect automation options. That is just a guess.
- milon 3y agoHaving a solid on-prem rack product to me is a great thing. I like IaaS services a lot, don't get me wrong, and I think they're the right pick for a bunch of cases, but on-prem servers also have their "place in the sun", so to speak :) I could present any number of justifications that I don't think I'm qualified enough to defend, but the gist is that at the bare minimum, I'm glad the option exists. As to why I'm personally excited: I enjoy the amount of control having such an on-prem rack would afford me, and there surely could be a great amount of cost-savings and energy-savings in many scenarios. Sometimes, you just need a rack to deploy services for your local business. I like the prospect of decentralizing infrastructure, applying all the things we've learned with IaaSes.
- bionsystem 3y agoIn the last 10 years and 6 different clients/employers I worked there is pretty much no way to run production on the cloud. Only 1 of them had some stuff running in the (GCP) cloud at all. Of all of the 6 infrastructures I've seen, only 1 of them is half decent, with 6 dedicated teams around the datacenter working closely together (by dedicated I mean, nothing is required of them concerning the core software product that the company develops). Network, Unix/Virtu, Windows, Storage, PC, and datacenter. That's 30+ people just to run a couple big datacenters and a few more server rooms. The service was actually quite good with VMs/zones delivered under an hour and most tech issues solved in half a day. The other infrastructures were either bigger or smaller, with more or less people, and were all terrible, sometimes needing weeks of email exchanges with excel attached to get a single VM. AWS was the dream everywhere I went for everybody. Oxide may be coming out with a product that will solve a LOT of issues. SmartOS/IllumOS has all the tech to be self-sufficient (virtualization, storage, SDN...), add support for networking and storage and you get a complete product that a handful of people can run (well, you still need a windows team in most cases but fine).
- SteveNuts 3y agoIt seems like the folks on HN tend to think the world runs on AWS (I'm not trying to say they don't have a huge market share), but many huge enterprises still run their own datacenters and buy ungodly amounts of hardware. The products that are on the market for an AWS-like experience on-prem are still fairly horrible. A lot of times the solutions are collaborations between vendors, which makes support a huge pain (finger pointing between companies). Or, a particular vendor might only have compute and storage, but no offering for SDN and vice-versa. This sucks because then you have two bespoke things to manage and hope they work together correct. These companies want a full AWS experience in their datacenter, and so far this looks to be the most promising without dedicating huge amounts of resources to something like Openstack.
- lijok 3y agoWouldn't a "full AWS experience in their datacenter" be AWS Outpost?
- mustache_kimono 3y ago> "full AWS experience in their datacenter" ... Including the bill!
- lijok 3y agoPart of AWS managed services offering. AWS Managed Wallet
- mardifoufs 3y agoIs AWS outpost truly a full AWS stack/experience? I thought it wasn't actually meant to be a "data center in a box" experience, but more so a way to run some workloads locally when you are already using AWS for everything else.
- PeterCorless 3y agoSome data products will run successfully in AWS Outposts. Others will not. For example, AWS itself can't run DynamoDB in an AWS Outpost. It recommends users to run ScyllaDB in DynamoDB compatible mode. e.g., https://www.scylladb.com/2020/09/15/scylla-cloud-on-aws-outposts/ https://www.scylladb.com/2020/09/15/scylla-cloud-on-aws-outp... Disclosure: I worked at ScyllaDB.
- throwawaaarrgh 3y agoIt's a mainframe. If you can't get excited for mainframes it'll be hard to be excited about this. IllumOS is the OS/360 to Oxide's System/360. (It won't get that popular but it's a fair enough comparison for illustrative purposes)
- panick21_ 3y agoExcept that it use the same standard CPU as commodity machine. Doesn't have much of the extra reliability stuff. It can go from vertical to horizontal scaling. The OS is open source Unix. And yeah its not like a mainframe at all really.
- linksnapzz 3y agoIt's a mainframe, for people who do not, actually, know what a mainframe is or does.
- kaliszad 3y agoI bet it costs a fraction of what a similarly powerful mainframe would cost. However I don't think the customers for each overlap that much. If you need a mainframe, you need one and there is no discussion about possible alternatives because there are none.
- mustache_kimono 3y ago> so the idea is vertical integration for on-premise server purchases? On custom OS? Why? Why would people pay a premium? As I understand it, re: vertical integration, the term is actually "hyperconverged". Here, that means it's designed at the level of the rack. Like -- there aren't per compute unit redundant power supplies. There is one DC bus bar conversion for the rack. There is an integrated switch designed by Oxide. There is one company to blame when anything inside the box isn't working. In addition, the pitch is they're using open source Rust-based firmware for many of the core components (the base board management controller/service processor, and root of trust), and the box presents a cloud like API to provision. If the problem is: I'm running lots of VMs in the cloud. I'm used to the cloud. I like the way the cloud works, but I need an on-prem cloud, this makes that much easier than other DIY ways to achieve (OMG we need a team of people to build us a cloud...).
- steveklabnik 3y agoThe terminology in this space is confusing, but "hyperconverged" isn't really what we're doing. I wrote about the differences here: https://news.ycombinator.com/item?id=30688865 https://news.ycombinator.com/item?id=30688865 (That said I think other than saying "hyperconverged" your broad points are correct.)
- mustache_kimono 3y ago> but "hyperconverged" isn't really what we're doing. This is fair, especially as it follows a discussion of how you're building at the rack level, which is more like a mainframe than current "hyperconverged" offerings. Recommend others read the linked post.
- lijok 3y ago> Why would people pay a premium? I would pay a premium just to not have to deal with HPE, DELL, etc
- _zoltan_ 3y agoDell's been nothing but fantastic for us (compute, not storage.)
- sarlalian 3y agoDell is a mixed bag depending on how well the individual region you are dealing with is doing overall. Things were great for us, but something changed and now getting good support for hardware failures has been a nightmare of jumping through hoops, time zone handoffs to other teams, and forced on-site techs to replace a stick of ram.
- adamnemecek 3y agoOne company making both HW and SW generally leads to really good, integrated experiences. See e.g. Apple.
- 0cf8612b2e1e 3y agoI am really hoping the broader industry takes note. By owning the platform, the Oxide team was able to dump legacy stuff that no longer makes sense.
- EvanAnderson 3y agoI'm excited to see how this compares to SmartOS. I'm pretty heavily invested in SmartOS in my personal infrastructure but its future, post-Joyent acquisition, has been worrying me. I really wish I did work for an org big enough to use Oxide's gear. Not having to futz around with bogus IBM PC AT-type compatibility edifice, janky BMCs and iDRACs, hardware RAID controllers, etc, would be so unbelievably nice.
- _rs 3y agoI had been using SmartOS for a long time but finally had to bite the bullet and give up. I ended up deciding on Proxmox on a ZFS root and am quite happy with it.
- geek_at 3y agothe nice thing about the Proxmox + ZFS setup is that it works and is even recommended without using hardware raid controllers. Less headaches either way. I recently wrote a guide [1] how to use proxmox with ZFS over iSCSI so you can use the snapshot features from a SAN [1] https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html https://blog.haschek.at/2023/zfs-over-iscsi-in-proxmox.html
- rjzzleep 3y agoI feel the same. I used a SmartOS distro called Danube Cloud for a long time and am looking to move and looked at Harvester[1] and OpenNebula, but with everything I know about Kubernetes(and LongHorn) I'm reluctant to use something so heavily based on Kubernetes. At its peak I reached out multiple times to Joyent to fix their EFI support for virtualization. The Danube team had similar experiences with them, working on live migrations for VMs, and a few months back I did a rebase of the platform image to a more recent illumos stack. Two of the fundamental issues with Illumos is that they don't seem to understand that they need to fix the horrendous platform build to get community support to keep up with the pace of development of other OS's. The platform build is a huge nasty mess of custom shell scripts, file based status snapshots, which includes the entire userspace in the kernel build. Basically if your openssl version is out of wack the entire thing will fail. Not because it has to, but because it was never adapted to modern needs of someone just wanting to hack on a kernel. It's fixable, but I don't see any desire to fix it, and even if that desire eventually shows up it might just be too little, too late. [1] https://harvesterhci.io/ https://harvesterhci.io/
- DominoTree 3y ago>> Oxide is pretty much the company I dream to work at, both for the tech stack, plus the people working there. Thought I was the only one :P
- yjftsjthsd-h 3y agoI mean, I phrase it as "my dream job is systems integration at Sun" (but Oxide is the living equivalent)
- rbanffy 3y agoSun had some lovely desktop hardware. They also had SPARC. I really miss Sun.
- jjav 3y ago> I really miss Sun. There's nothing in the world I miss so much as Sun. I've done many startups post-Sun and there's been good highs and many lows but, there's nothing like a true hard-core tech company like Sun. There basically are no tech companies anymore, other than Apple, but they are mass-market consumer oriented which is not interesting.
- rbanffy 3y ago> but they are mass-market consumer oriented which is not interesting. Kind of. Apple focuses their high end gear on creative professionals. Us Unix geeks have much more modest needs, which are often satisfied by the average uninspired Dell design. Still, Apple has a decent Unix underneath all that glitter. At the same time, there are almost no desktop-friendly Unixes besides the free crowd. HP and IBM have given up on the Unix workstation market eons ago. IBM’s POWER gear can crush the best Xeons and Epycs, but they have nothing to compete with the “good enough” low end. It’s a shame Oracle doesn’t offer Solaris on their cloud the same way IBM offers AIX (and Z, which, surprisingly, is a certified UNIX as well) on theirs.
- 3y ago
- jjav 3y ago> Oxide is pretty much the company I dream to work at, both for the tech stack, plus the people working there. Same for me. Oxide is the only company I know that I'd really love to work for. Similar (I think, observing from the outside) to Sun. That's what I dream about. Unfortunately their pay structure is such that I can't afford it, with a family to support. Maybe when the kid is out of university, if I don't need much income anymore, I can fulfill the dream.