de-duplicating

I may be mistaken but the issue the author is concerned with is de-duplicating breaking stories, not preventing the reposting of items from the long tail.

Main part of the idea was not creating a huge wish list, but rather just the de-duplicating part of the wedding registry.

So de-duplicating with a larger chunk size seems to work fine in practice.

So companies will have to figure out an efficient pay out structure, also with so many people in the same circles, de-duplicating lists.

There are some obvious restrictions such as requiring that the message be encrypted with a Diffie-Helman key (negotiated when the message is first transmitted to the receiver's network) to reduce the feasibility of de-duplicating millions of messages.

Fortunately the uploads were from a set of a few dozen static files, and de-duplicating the data via fdupes was able to drop disk usage by a factor of 20-50x.

As far as I can remember using inheritance over composition for de-duplicating code has been considered bad practice in OO circles for at least 15 years.

My buddy and I came up with a way of taking large cyclical object graph blobs of json and de-duplicating the objects for transfer over the wire.

So torn between a (possibly outdated) library, and code written for an, let's say unusual, runtime, neither of these two really appeal if I wanted to build a de-duplicating audio library.

One nitpick: It looks like the automated scraper is bad at "de-duplicating" syndicated (or stolen?) content.

More accurate: "new idea to the mass hacker nerd HN market." Solaris has had—for years—container based virtualization with local virtual network and de-duplicating file system snapshots you can tar up and send across machines.

De-duplicating might have a couple of downsides: 1) Not all good stories get taken up the first time.

De-duplicating the same could should be a tooling issue.