awgupta

6 ms·

1.
by awgupta 8y ago
This is a different feature than PiTR. We have both. They serve different needs.
2.
by awgupta 8y ago
that's correct.
3.
by awgupta 8y ago
That is a different feature (although a cool one). They provide the ability to run a query as of a point in time. We are moving the database backward in time (which matters for running applications)
4.
by awgupta 8y ago
Also, generally DDL statements like DROP TABLE typically autocommit.
5.
by awgupta 8y ago
You can also clone the database volume in Aurora and then just backtrack one of the volumes. That should help you ensure you have a version available for forensic analysis.
6.
by awgupta 8y ago
You're right - it is a lot faster. That's because we don't change any data blocks. we just mark a portion of the log to be ignored.
7.
by awgupta 8y ago
That's not correct. We made a change to mark portions of our log-structured storage as though they should not have occurred. It is a totally different approach than point-in-time restore.
8.
by awgupta 8y ago
As the previous commentor said, this isn't point in time restore or a change in our processes. It is a change to our database and storage engine to mark regions of log-structured storage as if they did not occur. That's why we can
9.
by awgupta 8y ago
We've had point in time restore for quite some time. Backtrack is different. It moves you to a different point using the same instance. Since we don't do destructive writes to blocks (it is log-structured storage), we can simply m
10.
by awgupta 8y ago
no, we don't perform destructive writes so we are able to simply mark a region of the log as to be skipped. this approach also allows us to move back and forth within the time domain even after a backtrack.
11.
by awgupta 8y ago
that's correct.
12.
by awgupta 9y ago
that said, I agree that approximate functions should be part of a modern database system. Redshift has approximate count distinct (based on hyperloglog) and approximate percentiles (based on quantile summaries)
13.
by awgupta 9y ago
The constraint for Redshift Spectrum queries tends to be between S3 and Spectrum. The graphs Jeff shows are in the Redshift cluster itself which, for the sample query he is running, is lightly loaded. We are working on upgrading our perform
14.
by awgupta 9y ago
it really depends on what you are doing. A large data set shouldn't be limited to longitudinal analysis. If you're storing every log record or every stock bid/ask, there may be times that you need to understand the specifics
15.
by awgupta 9y ago
The underlying tech stacks are totally different.
16.
by awgupta 11y ago
RDS just cleared HIPAA and BAA. See here: http://aws.amazon.com/compliance/hipaa-compliance/ Working on Aurora...
17.
by awgupta 11y ago
You can think of Aurora as a single-instance database where the lower quarter is pushed down into a multi-tenant scale-out storage system. Transactions, locking, LSN generation, etc all happen at the database node. We push log records down
18.
by awgupta 11y ago
We've made a number of improvements relative to MySQL, for example with large numbers of tables and the results set cache. There are some improvements on large tables and schema changes, but quite a bit more to be done in those areas.
19.
by awgupta 11y ago
That is the basic idea on such things, though distributed protocols become complex when you have to consider split-brain, multiple concurrent failures, etc which all can occur in large-scale events. Aurora leverages Amazon RDS Multi-AZ in t
20.
by awgupta 11y ago
it is a per-instance number. For those SysBench numbers, we are running an r3.8xlarge instance type.
21.
by awgupta 11y ago
Aurora isn't implemented based on either Galera or NDB.
22.
by awgupta 11y ago
For TPC-C like benchmarks, you can run: 1) CloudHarmony: https://github.com/cloudharmony/oltpbench 2) Percona: https://code.launchpad.net/~perconadev/perconatools/tpcc-mys... We've found