You need to be careful with AB testing. Sometimes your subjects communicate with each other, and are angry that different people get different web sites. Sometimes they dislike the B side of the test so much that after a decade or so using your service, they not only stop visiting you for two years, they also bad mouth you to everyone they know. (That last example was me, and Amazon. It took me months to realize that the egregious change they'd inflicted on me hadn't been inflicted on all their customers - the others got boiled slowly, with small increments of "sponsored content", rather than going instantly from a working search to one where 50% of results were irrelevant and labelled "sponsored", and another 25% were irrelevant but not labelled "sponsored".)
More relevantly, your A/B test proposal for Barcelona wouldn't produce results that compared status quo and ban all tourist flats. You'd have half as many tourist flats across the whole city, or maybe a bit more, and a smaller shortage of non-tourist flats to be inhabited by local people. This would be the same as if you'd banned unlicensed tourist flats (with enforcement), and then issued licenses to only half of the previous number of would be tourist flats. Except that your version has more per-neighbourhood distortion.
To get a real AB test, you need two cities, far enough apart that one can't easily live in one while working in the other. And other things need to be equal. Similar housing stock, similar incomes, similar levels of tourism.
I still don't use the company I now refer to as ScAmazon unless there's no alternative for a particular purchase. I doubt they measured that as anything other than "sometimes customers randomly stop using our services". And they are still offering me free trials of Amazon Prime.
Editted to add: this is, of course, analagous to my likely reaction if government health authorities announced untruths while testing some means of handling a pandemic in my locality. I can't imagine them saying "we don't know which is better, so in cities with names beginning with A thru M, do X, and in other cities do Y". What they'd say, to those on A-M, is that X is the very best course - while contradicting themselves in N thru Z. As we saw, to an extent, with individual countries and their non-identical policies. Every last one pretty well told its residents that their choice was the best possible.
Of course during the covid pandemic they announced a few untruths even while not doing much in the way of A/B testing. Sometimes they even knew they were wrong, rather than merely knowing that they were basically guessing.
You need to be careful with AB testing. Sometimes your subjects communicate with each other, and are angry that different people get different web sites. Sometimes they dislike the B side of the test so much that after a decade or so using your service, they not only stop visiting you for two years, they also bad mouth you to everyone they know. (That last example was me, and Amazon. It took me months to realize that the egregious change they'd inflicted on me hadn't been inflicted on all their customers - the others got boiled slowly, with small increments of "sponsored content", rather than going instantly from a working search to one where 50% of results were irrelevant and labelled "sponsored", and another 25% were irrelevant but not labelled "sponsored".)
More relevantly, your A/B test proposal for Barcelona wouldn't produce results that compared status quo and ban all tourist flats. You'd have half as many tourist flats across the whole city, or maybe a bit more, and a smaller shortage of non-tourist flats to be inhabited by local people. This would be the same as if you'd banned unlicensed tourist flats (with enforcement), and then issued licenses to only half of the previous number of would be tourist flats. Except that your version has more per-neighbourhood distortion.
To get a real AB test, you need two cities, far enough apart that one can't easily live in one while working in the other. And other things need to be equal. Similar housing stock, similar incomes, similar levels of tourism.
I still don't use the company I now refer to as ScAmazon unless there's no alternative for a particular purchase. I doubt they measured that as anything other than "sometimes customers randomly stop using our services". And they are still offering me free trials of Amazon Prime.
Editted to add: this is, of course, analagous to my likely reaction if government health authorities announced untruths while testing some means of handling a pandemic in my locality. I can't imagine them saying "we don't know which is better, so in cities with names beginning with A thru M, do X, and in other cities do Y". What they'd say, to those on A-M, is that X is the very best course - while contradicting themselves in N thru Z. As we saw, to an extent, with individual countries and their non-identical policies. Every last one pretty well told its residents that their choice was the best possible.
Of course during the covid pandemic they announced a few untruths even while not doing much in the way of A/B testing. Sometimes they even knew they were wrong, rather than merely knowing that they were basically guessing.
Interesting points!