Yes, the concept fascinates me too, to great extent. You have your thinking very clear @AndrewMackie!
I am completely sold on your description of the problem-side. It’s fantastic. Please let me ponder and speculate a bit more on the solution side. I haven’t given this enough attention yet myself, so these are raw thoughts off the top of my head, and may cover what you already wrote down elsewhere…
Convincing people to participate in a distributed network of offerbots from scratch, however, is almost impossible.
Yes, you need network effects for that. But not having these might not be as hampering as, say, leaving Facebook and in one go losing online contact with all your friends and family. There is less FOMO involved, and you can happily use OfferBots side-by-side with the old system.
The intermediate solution is to create a centralised offer processing system which is neutral and does not exclude any party on the basis of their ability or willingness to pay.
This is not really needed. A very large centralized offer processing system would be costly, and either needs to be subsidized or charge for its services which makes it vulnerable to a more competitive aggregator (if the protocol is not watertight).
The federated system will do just fine. The ideal end result would be a peer-to-peer network where everyone has their own offer processor, but that will probably never happen. Instead start with the model you suggested where “each person or organisation hosts (or pays to have hosted)” their offers.
The federated servers constitute centralization points, or Hubs, and the individual processors are Endpoints.
This is no different to how the fediverse currently works (and also Secure Scuttlebutt, which is mixed p2p / federated, and others). Note that there is always a tendency and risk for centralization to occur (like in the fediverse where mastodon.social
is by far the largest instance), after which that hub starts to create vendor lock-in, and eventually forms a ‘de-facto’ walled garden with network effects working in its favor (e.g. Gmail).
On the fediverse this is mitigated by:
- Culture and norms (currently there are no ads, good luck setting up an ad-based commercial instance)
- Ability of other federated instances to select who they federate with (e.g. exclude hate-speech groups)
- Ability of users to decide who they follow and who they block (hinges back on culture too)
Like the fediverse many parties are incentivised to run a Hub and provide access without cost. NGO’s, political parties, special interest groups, etc. On the fediverse you have e.g. social.coop
(for cooperatives), fosstodon
(spreading FOSS mindset), queer.party
(LGBT+). But also any other business, vendors and shops. They may cough up the cost, and incorporate it in the price of the offers (this will be like e.g. 0.001 ct per offer, like micropayment). Of course they can also have other payment models, like subscription-based pricing.
(Btw, note that the fediverse ActivityPub specification allows there to be a mixed federated and peer-to-peer model in the future).
Offerbots provide external offer processing by:
- […]
- applying processing (algorithms) to stored offers in order to filter and prioritize offers, and
- representing a subset of received offers to its owner (you) using any format (a web page, email, text, app view, app notification, etc.).
Yes, but also you set your filter beforehand to only receive a small subset of offers to process and filter afterwards. Your filter broadcasts your ‘default’ interest so you don’t miss out on interesting offers.
Interesting to note that in order to expose your interest filter you could broadcast ‘Offers to receive Offers’. This is confusing, so you might introduce the ‘Request for Offer’ or ‘Offer Request’ terminology (but it is still an offer type).
This also reduces network bandwidth and with all Endpoints acting together it is a natural filter for unwanted offers. There are all kinds of mechanisms to do this. While important information may be gossipped to become generally available (hosted on many Hubs and Endpoints), other offers will only be hosted shared after you showed an interest for it (similar to PeerTube, where you have a link aggregator to video’s and when watching, you become a node where others can stream from).
Interesting is that you might not want to share many of the offers you get. When booking a vacation or reserving a seat in the cinema I have no interest to share the offer, as the price will rise or the show is sold out.
[Offers] can be shared with other offerbots via the web and other protocols as needed (e.g. BLE for making and receiving offers locally).
Note that in a peer-to-peer system, in order to reach Endpoints that are on a private network and behind a firewall, you either need a server host within the network (what Secure Scuttlebutt is doing) or you need to do NAT traversal (what e.g. Dat Foundation is doing).
Offers may be made publicly or privately with any party and are signed for authenticity.
This means that in many cases they will be both signed and encrypted, or partially encrypted (e.g. having a public header and encrypted PII).
Offerbots, therefore, need to operate at a different layer to browsers. They will:
- use the web to send and receive tens, hundreds or thousands of offers with offerbots (through web APIs),
- filter, score and rank received offers (according to its owners preferences), and
- act as a publisher by generating maps (web documents, emails, app views, notifications, etc.) which represent offers to their owner and their choice of other people (through a browser, email program, app, etc.).
Up to this point, and reading the above it is clear to me that there is still a huge market for aggregators to exist. But I agree with you that parts of the problem aggregators cause, can be mitigated. There is a lot to analyse still, and that takes a concerted effort. We have to avoid having commercial interests go loose with this whole idea, before we have a change to fill in the gaps, or in a couple of years we are at the same point where we are now.
SWOT
Strengths
- The clear description of the problem makes it way more likely to find people willing to participate in finding the best solution.
- A large part of what’s needed solution-wise in terms of code and knowledge is readily available on the web (open source, information resources)
- When solution-side is also presented well enough, many people of the right mindset will be available to drive this forward in the proper direction
Weaknesses
- The idea is out in the world now, and the work is only protected by @AndrewMackie’s copyright. I don’t know if that is enough protection to hold off unwanted follow-up.
- The linked data initiative (and its predecessor ‘semantic web’) do not have wide adoption. The biggest player is Google for SEO (via the Knowledge Graph).
- The data models and ontologies to accurately describe all the offer types and metadata do not exist yet. The tooling and effort required to easily create them should take priority.
Opportunities
- Unlike browsers that evolved to a state where e.g. tracking opportunities (cookies, browser fingerprints, javascript) are hard to mitigate, the fresh start of OfferBots can close all these loops from the start.
- The fediverse is the perfect and natural home to testdrive all of these concepts!
Some more technical points:
- There are standards like Solid Project in the making that allow you to keep control of your own data (via Personal Online Data stores, or PODs), or as
@tag42git
described it to me “a way to set read and write permissions for the giant global graph, the web”.
- Note, I have some reservations against Solid Project, among others the commercial motives (it is more an ‘open core’ project), but the idea is solid
- There are standards that allow decentralized anonymous, pseudonymous, and validated identities to be defined. Other standards define object capabilities, and validated claims that can be useful.
- Encryption technologies have come a long way and provide opportunity to creat all this with a true privacy-first approach
- Decentralized technologies like fedearation and peer-to-peer have significantly in recent days, partly due to corona crisis, but also because of the ongoing and growing techlash.
Threats
- The Hubs become aggregators that do the filtering, scoring and ranking before you have a chance to do that.
- Their scale, speed, processing power and reach they already have will distort the landscape of offers
- The linked data in the offers only makes fine-grained profiling and microtargeting more likely
- Big players own their own proprietary semantic data models they could use to compete with (like Samsung with viv.ai)
The SWOT should be further elaborated, but I leave it with this for now