parent
3b9b00acd6
commit
290d51b92b
@ -0,0 +1,53 @@
|
||||
// Fish (salmon)
|
||||
value_happy_salmon_year_in_human_qalys = 0.001 to 0.01
|
||||
value_tortured_salmon_year_in_human_qalys = -(0.002 to 0.02)
|
||||
value_farmed_salmon_year_in_human_qalys = -0.02 to 0.01
|
||||
// ^ purely subjective estimates
|
||||
lifetime_salmon = 2 to 3
|
||||
// https://thehumaneleague.org.uk/article/how-long-do-salmon-live
|
||||
weight_salmon = 3.5 to 5 // kilograms
|
||||
// ^ https://www.wildcoastsalmon.com/salmonlifecycle
|
||||
calories_per_kilogram_of_salmon_meat = 1700 to 2100
|
||||
calories_salmon = weight_salmon * calories_per_kilogram_of_salmon_meat
|
||||
// ^ see <https://www.quora.com/How-many-calories-are-in-1kg-of-salmon>
|
||||
|
||||
salmon_estimates = {
|
||||
name: "salmon",
|
||||
value_year_in_human_qalys: value_farmed_salmon_year_in_human_qalys,
|
||||
weight: weight_salmon,
|
||||
calories: calories_salmon,
|
||||
lifetime: lifetime_salmon
|
||||
}
|
||||
|
||||
/* Fish Welfare Initiative
|
||||
|
||||
Sources:
|
||||
- https://www.fishwelfareinitiative.org/
|
||||
- https://forum.effectivealtruism.org/posts/T5fSphiK6sQ6hyptX/opinion-estimating-invertebrate-sentience#Peter_Hurford
|
||||
- https://forum.effectivealtruism.org/posts/Qk3hd6PrFManj8K6o/rethink-priorities-welfare-range-estimates
|
||||
- https://nunosempere.com/blog/2023/02/19/bayesian-adjustment-to-rethink-priorities-welfare-range-estimates/
|
||||
|
||||
Key simplification: assume that all fish are salmon. This is inaccurate, because salmon is a very particular & expensive species of fish. But I think it's ok to start with. Later I could easily add different species.
|
||||
|
||||
*/
|
||||
|
||||
fish_potentially_helped = 1M to 2M
|
||||
shrimp_potentially_helped = 1M to 2M
|
||||
improvement_as_proportion_of_lifetime = 0.05 to 0.5
|
||||
|
||||
value_fwi_fish =
|
||||
fish_potentially_helped *
|
||||
improvement_as_proportion_of_lifetime *
|
||||
(value_happy_salmon_year_in_human_qalys / salmon_estimates.lifetime)
|
||||
|
||||
value_of_shrimp_in_fish = (0.3 to 1)
|
||||
// ^ very uncertain, subjective
|
||||
|
||||
value_fwi_shrimp =
|
||||
shrimp_potentially_helped *
|
||||
improvement_as_proportion_of_lifetime *
|
||||
(value_happy_salmon_year_in_human_qalys / salmon_estimates.lifetime) *
|
||||
value_of_shrimp_in_fish
|
||||
|
||||
value_fwi_so_far = value_fwi_fish + value_fwi_shrimp
|
||||
|
||||
@ -0,0 +1,27 @@
|
||||
General discussion thread
|
||||
=========================
|
||||
|
||||
Do you want to bring up something to me or to the kinds of people who are likely to read this post? Or do you want to just say hi? This is the post to do it.
|
||||
|
||||
### Why am I doing this?
|
||||
|
||||
Well, the EA Forum was my preferred forum for discussion for a long time. But in recent times it has become more censorious. Specifically, it has a moderation policy that I don't like: moderators have banned people I like, like [sapphire](https://forum.effectivealtruism.org/users/deluks917) or [Sabs](https://forum.effectivealtruism.org/users/sabs), who sometimes say interesting things. Recently, they banned someone for making a [post they found distasteful](https://forum.effectivealtruism.org/posts/AAZqD2pvydH7Jmaek/?commentId=CDPS2JQKziWsWg73D) during April Fools in the EA forum—whereas I would have made the call that poking fun at sacred cows during April Fools is fair game.
|
||||
|
||||
So overall it feels like the EA Forum has become bigger and like it cares less about my values. Specifically, moderators are much more willing than I am to trade off the pursuit of truth in exchange for having fewer rough edges. Shame, though perhaps neccessary to turtle down against actors seeking to harm one.
|
||||
|
||||
On the other end of the spectrum, [The Motte](https://www.themotte.org/post/430/quality-contributions-report-for-march-2023) is more spiky (waaay more spiky, warning if you click that link: irreverence towards any and all sacred cows), but it is also more unfocused and less forecasting-adjacent, and I don't know the community there.
|
||||
|
||||
So I thought I would try hosting a discussion here. Note that registration isn't needed to post a comment.
|
||||
|
||||
### What I expect will happen
|
||||
|
||||
I don't really know, that's why I'm trying this!
|
||||
|
||||
One salient possibility is that I will go down the same censorship slope as everyone else, where I start out being very in favour of uncensored speech and then decide that that is not optimal, after all. But I'll do my best to not censor stuff in this first thread.
|
||||
|
||||
<p>
|
||||
<section id='isso-thread'>
|
||||
<noscript>Javascript needs to be activated to view comments.</noscript>
|
||||
</section>
|
||||
</p>
|
||||
|
||||
@ -0,0 +1,31 @@
|
||||
A Soothing Frontend for the Effective Altruism Forum
|
||||
====================================================
|
||||
|
||||
### About
|
||||
|
||||
<a href="https://forum.nunosempere.com">forum.nunosempere.com</a> is a frontend for the [Effective Altruism Forum](https://forum.effectivealtruism.org). It aims to present EA Forum posts in a way which I personally find soothing. It achieves that that goal at the cost of pretty restricted functionality—like not having a frontpage, or not being able to make or upvote comments and posts.
|
||||
|
||||
### Usage
|
||||
|
||||
Instead of having a frontpage, this frontend merely has an endpoint:
|
||||
|
||||
```
|
||||
https://forum.nunosempere.com/posts/[post id]/[optional post slug]
|
||||
```
|
||||
|
||||
An [example post](https://forum.nunosempere.com/posts/fz6NdW4i25FZoRQyz/use-of-i-d-bet-on-the-ea-forum-is-mostly-metaphorical)---chosen to be a link post so that you can see both the article and the comments---looks like this:
|
||||
|
||||
|
||||
|
||||
### Notes
|
||||
|
||||
- Clicking on an article's or on a comment's timestamp directs to their location on the original EA Forum.
|
||||
- I am choosing not to have a frontpage because it would just add additional development time to a hobby project, and because it doesn't support my use case. My usecase is just being subscribed via rss, and displaying forum posts which interest me in this new frontend.
|
||||
- On the topic of rss, this frontend has an rss endpoint at [forum.nunosempere.com/feed](https://forum.nunosempere.com/feed): it piggybacks off the [ea.greaterwrong.com RSS feed](https://ea.greaterwrong.com/?format=rss).
|
||||
- Other alternatives to the EA Forum known to me are the aforementioned [ea.greaterwrong.com](https://ea.greaterwrong.com/) as well as [eaforum.issarice.com](https://eaforum.issarice.com/).
|
||||
|
||||
<p>
|
||||
<section id='isso-thread'>
|
||||
<noscript>Javascript needs to be activated to view comments.</noscript>
|
||||
</section>
|
||||
</p>
|
||||
@ -0,0 +1,57 @@
|
||||
Review of Epoch's *Scaling transformative autoregressive models*
|
||||
================================================================
|
||||
|
||||
We want to forecast the arrival of human-level AI systems. This is a complicated task, and previous attempts have been kind of mediocre. So [this paper](https://epochai.org/files/direct-approach.pdf) proposes a new approach.
|
||||
|
||||
The approach has some key assumptions. And then it needs some auxiliary hypotheses and concrete estimates flesh out those key assumptions. Its key assumptions are:
|
||||
|
||||
- That a sufficient condition for reaching human-level performance might be indistinguishability: if you can't determine whether a git repository was produced by an expert human programmer or by an AI, this should be a sufficient (though not necessary) demonstration for the AI to have acquired the capability of programming.
|
||||
- That models' performance will continue growing as predicted by current scaling laws.
|
||||
|
||||
Then, note that today's language models are in fact trained to mimic human text. And note that the error that their training method aims to minimize can be decomposed into some irreducible entropy part, and some part in which is due to the model not yet being a good enough mimic. So then, we can ask, when will a language models' loss be low enough that it is approximating human text enough that we can be very sure that it has acquired enough human capabilities that the model will have transformative effects?
|
||||
|
||||
For example, if a model is able to produce scientific papers such that it takes multiple papers to distinguish that model from a talented human scientist, then this would be enough to conclude that a model has acquired the capability of doing scientific research as good as that of the talented human scientist.
|
||||
|
||||
My high level thought is that this is an elegant approach. It warms my blackened heart a bit. I'm curious about why previous attempts, e.g., various reports commissioned by Open Philanthropy, didn't think of it in some shape.
|
||||
|
||||
One thought though, is that the scaling laws that they are referring to do have a large number of degrees of freedom. Their shape is:
|
||||
|
||||
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
|
||||
<script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
$$ L(N,D) = E + \frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}$$
|
||||
|
||||
and we notice that there are four degrees of freedom (A,B, alpha and beta), as well as the choice of overall shape of the formula. The E parameter represents irreducible uncertainty, but also seems to be only empirically estimatable by estimating the other parameters, and thus also seems to take the role of a dangling parameter. I am reminded of the quote "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk". I don't really have strong opinions about whether scaling laws are overfitting on past data. But I do think that most of the uncertainty is going to come from uncertainty about whether scaling laws will continue as they have. Still, having a conditional forecast based on scaling laws continuing is fine.
|
||||
|
||||
Anyways, then to flesh out their approach, the authors need some auxiliary assumptions. Some which I could notice are:
|
||||
|
||||
1. That at the moment when a rater exceeds some threshold of certainty when trying to catch a model vs a human, their certainty will be very close to their threshold of certainty.
|
||||
2. That human language is ergodic.
|
||||
3. That a human trying to distinguish between a mimic model's output and that of a human would exhibit a particular mathematical shape, where a human moves in the same direction as an ideal bayesian reasoner each step, just more slowly.
|
||||
|
||||
Honestly, these seem like reasonable analytical approximations. But I wish there had been some discussion of when language cannot be ergodic. What I'm thinking is, that it does sound like a reasonable approximation, but that I haven't thought much about it, and that maybe there is a catch, like texts containing novel scientific discoveries maybe not being ergodic? Also, ergodic over what domain? The domain of texts which could be written, or will ever be written? But then language models don't have future or counterfactual texts in their training data? I don't know, and I haven't thought much about ergodicity either, so I'm uncertain here.
|
||||
|
||||
I also found assumption number 3. to be irritating, because the model assumes that human judges have a "slowdown" factor in their updates, but then humans could notice that, estimate their slowdown factor, and then update faster. My guess is that instead of, or in addition to a slowdown factor, humans have an error rate, where they process evidence as pointing to one side that an ideal reasoner would process that same evidence as pointing to another side.
|
||||
|
||||
I also didn't really like assumption number 1, about the endline uncertainty when waiting for enough evidence to exceed a threshold of uncertainty being close to that threshold. That assumption wouldn't be the case if, for example, most of the outputs of a model where nearly indistinguishable from a human, but it infrequently had some clear tells or artifacts, like inexact human hands in some arts models or unnatural moves in chess or in go.
|
||||
|
||||
But overall, despite being irritated by these auxiliary assumptions, I think they are generally a reasonable move. I don't really expect assumptions 1 and 2 to introduce much error. I'm a bit more wary about assumption number two, because as I said I haven't thought much about ergodicity, but it seems respectable to add more analytical clarity at the cost of wrapping the whole result in another conditional (... if language is such that it can be reasonably modelled as ergodic, then...). Also, the alternative might be to explore each tendril down the forking paths road, but this would be too effortful.
|
||||
|
||||
To conclude the analysis, the authors need some forecasts about how indistinguishable AI-produce texts have to be before they are transformative, operationalized as how many tokens one would have to see until one can be confident that they are in fact produced by an AI. Their given forecasts don't sem particularly objectionable, thought that I haven't had enough time to sit down and examine them carefully. More generally though, once the authors have outlined an analytical approach, it doesn't seem that difficult to commission forecasts to feed into their model.
|
||||
|
||||
As for suggestions, my main one would be to add more analytical clarity, and to be a bit obsessive about what analytical assumptions, key assumptions or otherwise, are going into the model. Maybe have a chart. If one does this, then the shape of what the paper produces is, I think:
|
||||
|
||||
- A conditional forecast
|
||||
- of an upper bound of compute needed
|
||||
- to reach transformative AI
|
||||
|
||||
The forecast is conditional on scaling laws continuing as they have, and on the various analytical assumptions not introducing too much error. And it's not a forecast of when models will be transformative, but of an upper bound, because as we mentioned at the beginning, indistinguishability is a sufficient but not a necessary condition for transformative AI. The authors point this out at the beginning, but I think this could be pointed out more obviously.
|
||||
|
||||
The text also generally needs an editor (e.g., use the first person plural, as there are two authors). As I was reading it, I felt the compulsion to rewrite it in better prose. But I didn't think that it was worth it for me to do that, or to point out style mistakes—besides my wish for greater clarity—because you can just hire an editor for this. And also, an alert reader should be able to extract the core of what you are saying even though prose could be improved. I did write down some impressions as I was reading in a different document, though.
|
||||
|
||||
Overall I liked it, and would recommend that it be published. It's the kind of thing that, even if one thinks that it is not enough for a forecast on its own, seems like it would be a valuable input into other forecasts.
|
||||
|
||||
<p>
|
||||
<section id='isso-thread'>
|
||||
<noscript>Javascript needs to be activated to view comments.</noscript>
|
||||
</section>
|
||||
</p>
|
||||
Loading…
Reference in new issue