From dd5559d09926c2802adb2b05cfb993212d916283 Mon Sep 17 00:00:00 2001 From: Nuno Sempere Date: Sat, 23 May 2020 19:53:27 +0200 Subject: [PATCH] Forecasting Newsletter for May draft --- ea/Forecasting/AccuracyVsConfidence.png | Bin 0 -> 53572 bytes ea/ForecastingNewsletter/April2020.md | 18 ++- ea/ForecastingNewsletter/Header.md | 19 +++ ea/ForecastingNewsletter/May2020.md | 199 ++++++++++++++++++++++++ ea/ForecastingNewsletter/Template.md | 66 ++++++++ index.md~ | 122 --------------- test.md | 1 - test.md~ | 1 - 8 files changed, 295 insertions(+), 131 deletions(-) create mode 100644 ea/Forecasting/AccuracyVsConfidence.png create mode 100644 ea/ForecastingNewsletter/Header.md create mode 100644 ea/ForecastingNewsletter/May2020.md create mode 100644 ea/ForecastingNewsletter/Template.md delete mode 100644 index.md~ delete mode 100644 test.md delete mode 100644 test.md~ diff --git a/ea/Forecasting/AccuracyVsConfidence.png b/ea/Forecasting/AccuracyVsConfidence.png new file mode 100644 index 0000000000000000000000000000000000000000..4137990a23a43d07f9c5b086f6d0792854351610 GIT binary patch literal 53572 zcmYIw2RK)6|G)MUlAY{~>`~bwGb@!)D1=1zo-I2G$x4c%BqPeEj3g^5va^$&z5lQC z?fG5*=enNfDdBU@x$pb^ey#TjKBJ|we=oyc5)zXAYO0E7Nl3QRkdSQg-MtfE>DhhG z1OK<(NkQ%WZv4l4_thZ$o!(hV-}#)Kg|nN9qdAGCt=$cCekU_Wb8}m#Yj)04+sb80 zNLWbJ6y?vmKORr?aHnpIOY1FKP4#-gyXWo&jxDAKb{}Hj`e`ep+3mJp_b0?%mzLC^*dDOey=vJoPEY@(6Ekc!pGWhE5>@epo_=6%_grSV8 z3G2VFetuA)@ZUGoK4A6t$G@p^M6#aRyN~!bRgO~uR5btnglYgO58uBZCZ#{CPy9kf zHhPig*$@8vqsZ0&{|oK!;bUk6Oz5fx9BhssI-anAGsdoyVs&>g3+4j}taA4&v6FRzpTbX0S4z9(AjoMa?dPkiX zR?Fwz9mte)R-!L6rlhAE98YMl@U_kF)S;?gyVMmZ9++zNb7CR)NzA;#?;_ic9*f*( z>FLXdW=c)y)}JJDSAK9~`C{txzJIeQGhv8wk++O?$W@%tQpU`If2L48OFaCYcfXfr zOZ~vkvTZldAN5b>7H1@*4BjpfGnQd#%fI>Y_N2{Ake2w(Oci##bl2O~S-MoeaxY`*`mfxB<_8yTZU}DcO)Pli z9+2!EpW0_HUOzDIx@usa|DMYsgL-V0&Rwq6jpvKajk;F1>pt48v!vc!Bw=A;b8~Y- zE`Pq~eqon5aaP}u^;-O=eS#$?&CJ*ZV)o^ChBa7Jjygx4e(`%^LP=M5%~wto!W#ooX-6DyHao%QDNz$Xqoy7xvrmE-@A)$OF` zaYd+7?A|`Ty1umTd!I?>%V5u;*Cf0y1Hx; zyGKVyN$>%ZQc_k{Rv)Jpqaq^YEi5=oOG`N@_u#s?w4YQ>Ev8s8`1|``wX{5^r>AoH zGP6KT=t-8Fbwg}%8fVX*Rgh+~oa!mXKPNQhW@g^UmA16Cshm2sEhPDCTU$+4@IfmZ z8;&@QWGe>zq&{CjgN0drtS}ztui53@JMTP}^@+bK5}eHKUfSc!_SH6jz_CzOU7ejW zm}0L#ou~yV$^86$UVeT{XQ%qbi}B0eQc_Z*V`J#zE{HqgJfGee^r!VNXJ^P9Ze{gnlJGj@MB&cQpPlhexX=&-ex!-yr_&gN_1sp3p;k_J{PvK{dB z^3r_9yOR%_{1Goxc)~U{+){fhl@e>527bGy$-v_3E$yAp@82iGRpF#!aqa8`Yp+rU zR?2H>kp(Mmy{XEfAkN6i5Z3Q3iHdH>2shZGM6aU-an%q19Mwl&_7^q>W&;C{8sKhX`(wJ?+u* zN2B{Kt$JNtSvjcxb!(os%dL;#cA}B2!ZQc5{r{{jRZ5sA?Qj-c>1dEhZ!rGcQY30+hGNwu)d?q$#%eQZ)WLHfo ztgWrtaYV2;e0^m#H8rorD@!wBS^MWbt*&2ZVRiGK<=) z-^Rt9jcO)biCxVW@nK{Z_cFcgjgs3kS~WKBI9yME`yiyY zKW<6W7PajXA9s~%zV>i#&e_w`6Z`M^ixh+mf*wdh%j9o9c1E}KN2-+99|HZd^~TbeB*$iO$e#%ynJS(z-!xqCq$ zWpNmdjfpLYCyrGqDlUGNl~pt3cp$u{VawFd9J7ZHAEL9UzBqeVffd&&?l>4E5z9%r z2Xz2Ha!o9xcj|=Hjie^aixgG$0t{i4q1E-bu=eM!;esPj)yBp|t4|5hJufakgzX!0 z89S1Ll2ueRN}o^J(lSjVigf24XJ_YM^PXstkE?fX-6A3?D&aItiXtSuOH*f;qmz?n zGfU&tjC=hSjl4K)Kxrm&6q~vEd6V3BoSN--Z{j z(XZ!I!4Zm3-JaGG+%zIO_N21!HA65W(!KFwl;Z7 zs%A=leoRk~@j3nHXL+!6+3hylw{BTkSrH=LhH9P6ZE?v(obj=w%ifzOS+2a7Yv$Z` ziv-&WJ8r;P(#p=RW@I)ZAwm7zxhNh!oS^trTr{fqIw$p;H*YKoi;IhKvHd8J7QN=U ze|KpT6sd%A>VsS#@~*tg%R~LM7iX+86~<+vPc5!CY#d<-i+uD*Sz9}N*{k`uQu*@N zQ{*!rwviit>RJDD>LOnz1@OVSR1K5Wg$e29!Kh&pmiiFWM<&O@E@?)mr^hFS(%(Nb zijtJK6McxZ^24Zeyr}Wg^Mm!ZwI78@yKM8@n}fzKHe_8mw{|7fS@?m0=EDo3>9G#M z_b)$g!8hmbb6_g&wk7MiDPcxwerH3S-}EMBuDnSy?eGm<-5Jl)_4F$_hz&)T@#^csxIOSNVD!PYO%^32AO8D6BU36VE@%G0qz zwlls1Ma$D}TW{}j{lLlc)!g{LUGdY^5>;n!7rb}h*)RLXd|SOIzvBD(_245UuXj`y zIkQ^)?sr4o!U5?C6xVv#va&HSQ&9NWyfo8ALS?N7+xyYJu~PE)8ElIAEu-U$S{Y3S zyGdtC^%vfjiL$!0M+8Z@?~SazT0V4o%+a5M7k&Gh*!7D(RdFVMVH*r$hkF`tb619t-e!AkN5akqXla`Ka{7JEogL*?o*taKB$S6Dx zUR=rAT*+Erp{@02^{=MBou0mZ>n-15=UXoPzQ2l6b%!pklT?Z9^?v)Dg(1gf{j88Z zKh4HCxe(o*{TAEg@*iD}Rj7`iE2LO7FMXeVJ*ZrUDO^H`^!>uoM)og9GP$1kyKD7} zvsP67;>`5PbPQ>a)Kw+r+x#$M8e4ZArfzn=3x=TNl&0_uA&KW;pJ#zQeuw zVgCBqFF{G;F)#YxW+K_+BdZ!~{5u(6YlqL%jlIig`aI#9FhqUlwR`Iw*89vsKa$Lz zK9FdOFKZh+ru4jJRO}61Yj4P3{aw7TuN{2Pl4ZoSPQ6*QK0>iOc9E8JXH$`#gN${%V&c1%T>aY5pMReye5)nC zbMG>cf$u@yv3GBLQs+yz4h|2;Pw6`LshT+WKOK`~3a>Fdm(RwK=(5w=ts1g~hOk<@}xXNz=lITE7H`&Psjk>v!x~_5T|qoubaaXc#he}dXlIT+mVfURKKpO_r zRdhB=E~)-|NGy-v+}%|~a18;$3SnJGhyR`&_uh|JzIJ`%Cdf#gI@$zE5WCSmUNRoO zo$}j5>Faa7p9_(qb;+Z)jOwCX;`;)7r?|MdET4xuRAcM10BQgVDMx2ha7rXKx#=_} z5R|1*Oe;zDhO$nyq4-&SeTt86zMHGPvvV!lEiEl(0hQDTcE7n55peLprIS^U32IY2 zq@*|&ZOG{O$69{7!WYyX+kB_BC5Ombj+$`h5tMlX(ea#1&A#&SwuxXus+Gm>LG|uT z*JloN1V{Qi5|kPl3O=>3;;4|@q~XGNdu3yyRKwS%6Fu(iyn2~~-(Q{MGc2(GS;v_y z8!}zNMjx?z-}sFCivM5gt%^DieNCP!Z9jRyTbZC^ci7L8s|pZLfu*YYMA^*#^SZ|d z`nP@m@xx(hx=%jEz}Pl})t_4F8KCWr&i8$@%jo?YNwSHzZ*Tq8R7m~j&!20yw#scT zxSoc)nrxNk78a>_SMc=qSzKYF8&e$euJP?C6c{f0DrdMnlvnkI#2^L~kVdhL^njP~ z)kAroVwu&L1M%Pi;sZ>CfEPCYWNm!OQx!XQEXT5m35#8?D8Gx0k%gW8j=MC`q}tot zu@)75*>O7OK|Q;J9`@ag)0{JHc1biUcI|U59-Ww|w#XIs_|h`W?*h< zCg#7h#uKV0CTI|7gFkaE!wcum=`g*Z#B|-Nd)LwP`S%)naKpPQc-Y=y zp1@1Bjg6`A-bMGWPr6C3O;@g;En1wPk7K(!Tp#x#DeBc89TBWC3GgAa?-J`lF0DZo zxs?OGt38XKKY#wUSiUJURkkY7XCDhhXwmefp*~I&CsulOJQrYkx^&7X9H_LYsOZ{_ z8`UGTGgF%zQxFdT+caPQtzq`7Z#dCgk8LwLK3-x&_NXWm1ku5u(dQVuzyFSyy6Oyr z+@++=SwG6%+wBVt86_O=h8;Tbt+n;jbG4|QnwBz*yJti_Z5qhR!wGhu zfbsYG>z8^S!w0A^`%r->CUb)lgVGfa+?T?-f|MA(k3Cg!o4rtcaJLuw2|Gvxe3G44S1(|vF>{dg<2~nRF=d!=iS@yrbvO*L% zt**WgbN~k{aN91jGe$;XCUoOng@7)nZ`|N7U+tbcqZ@sah2+D^{Fr8FU}YaT(Gy{_ zPY)hF`h*g@d;3;E%lP)l_N)yJ4RitWLPX8|kN6nDgSqi-1$pwd2H_kJ*Am=*f=7ssm4Uc%Y0Vd|GY}0lk=69 za+`c!R1}|JTeN4%>Ei7d>M{7lgEwPUS(#c78fTZC$kZ!qoLsi?&--BCY0gN z_{;O>cd>19t~IljEjQTxt-HJtnVx<$Bs6qp*?aiy*vvo;4=*n`Am~PYtQ-W2)>dV( zYRDyX3k%=6y6S7AxFkIOa=7*SoR}SH8UZ%P9dlDvOf4P^Rs;h=jX6vg@G(}{?4~xS z&n|MN!5?~;kAQS0ZLW=gyB-KrCNMotVePoEA}Q{@BotusATCbLz@YtsaZ~&%^oaAiGU`Y6W&J+M!g9(}ouiH0H0C0j z!&LdEESk7;W8;@ETYu;Cef#?Lq=SPn|E zACRfi)4%of*b4Y8dCM+LbOt7KgR2Rn={mKk9*s8r@{os$ipsw8M!xWk4qWz&{QSkY zGIqMUfqV&h>FL;Bw?F=R@Yqg_UFGE#9L~w|O<&uhQBIH{7Z;cD&iruxOYgsSbX@m5 zy7?mLlVg9?9%H|Cm%V%Ux=(-H1^zk`Q@(a&bH03|E!X#8nDXeKKSdVjvlJcQm>CUd zzc@=sM>rR`p7Ynywp%E0m49bv8+w=5eJ2YB$b8oZm~;I$eZjv$%ep+r-T)RnD4r@i z6jP_<nHeRk^3QS3J@NH@T8=Z~mFsdB`SLnu>Z8}E1*IzQItv@~sk(`1l$?LdCF z>mi)Kt7ENiG@eMe=Q$n?x!r9$(kL|rc1=8IXh(haKdDGSkJW8ruek@EAWG(T~(GuTUCi%;o`n);>_ODTf(LP?moz=!Gce@vy0p`zmrEmVE9dbk7;e>5n>%t8Pi|C zMsp>P;?y+ltMDK1E{_-rJ)W5nI!I6MNwgr4mn}$*(O3ZwZ-n>ze*N=Jq`nAWz zyEdZGEOxswkF{l4{`vl@P4W5;r53Tc=IbPoz7~7^k`krdzTrGwxbSFnkI|Vk_Kg$q z6{kA;?7wz(X$a9l7A3zJe7Cy4Ms7nGVj46ay@1((0s8$Qma|{`5UB-uS^J5xtU(cTd_B| zqkE&fC>mGc*d()f#?a7a#nmbvkQIF-;?X15#>194oBw2vZ~!96Vvn`B3=9lXdt;B$ zG@*Tb{_>@w&wjdMpWp)x&gJ6rRVge%XTBZrRDS0(@_oq~lk{4g93uBy+IH*Y%^b6u zvw(8)@*mCVGnoC>9G-ao;Ry)|sb3ANHuYYcsTu#7b0;KZuWiYU@@(Aoc#De>WwVJ> zLUujUWQQdcm6Uc6zYGWnu&A_Gw3H!&CRF|#Giui8ehYaVTL-p1)Z|BNN|ksm&Xutc?!K*3f4kWs-J(aH7|{#hK9$ z6*f-JPZnHan2hwbU*+Wx@utG|z8AZZs4JP70>7dH z-lDep0x5|A<&iN-fA(4IR>5Y&nePYUf&hp*_R%a{=2HkYFN~K7S`u}!lGp8bIU@}B z1x-9vxzE@K6nhnT3vlHe>ZxAYU+PrtBwMnwiof=SJDVf^XlM8sa8nii?j;Z02Cgj@ zbLB3Dqh#X4N00ta7Ei%^f;0)O`K{B47L(Vw@Wz1L#vQZ?ovIdUX*aUp4A zj*DnokN6BKGL4Ft7X3EeHYR#EAlvKb+aAU_>Flh-Ar<#o5km#eb0)#gjfA`?%>=XP z*Wzl{%HE?9hPAs@WN(2oKS(=f-D*GmOP*8f@#jhR*;ydzn|02x&Gs5kI$k}m+gsg*%fXN508<# zKkt0sgX3(yb<^M@0MN@8v1@kg~IV0 z9aRUlrRDi(1|TP3>uke$VSbR9s0o7%xbJytYQW|i$dB&k{TIZRmb=*IGFdpnM6gS? zh=O0|2LQi-wxL-O1a@g-vAn6TGEj#Kn(jRlItUDLeT7PtPaOt6%?w6qgEc8b#zynO z*-AkFgxm|fjap)2@@&+AFG5wB)>Ot=hYAc_|J`-!OYd!-y?PalE43}1zX0l3TUR$W zBF0lc5gAd9E(PKa=*~zs`|A$**^3uXFSF;T#;E7o7 z46cQSW09NVGIjwih#*;$U4`5s6(A-&_{}4;Fm#{+q-J|;ERRTd&T&^)SLfyBow;hXSn=yBaT=dog(^GNzQT7MGvW)qUbI zOwXzPL_gMio0zbViYvA*bi!0tPQ+^P(D zWgW3y6)Y{eHiydBXn`2P?4M_3)RV@HxfT~r7L7F;NI+dEER17b`?YJBh67Z$z+qsi zkKFGw_U6*4wjYqiLz#qL=R0$^>RW4biV8L_2ebv|jp;yu zWF{0gZJILcjG;!2{`J+g8TSD8xGXF>@#0-$=}h@ zF&4EH)JA%Ju4VYiB$%=eHUbCb7NH;G%Zec_quYgA-g#j`cgG-j;G5BZvdx9z(4qD+ zA1S|;mZYy@1bdh)TeStUX(-rVdB9FyWRCRS;pg0Y31GR}yD1BTcmBeKvuDm+S*Rg2 ziY)rqRm%@jT3B%11V5{*Gy8aZ2WTV=*H^Fh#MI5dS`9BX7SxuRe`xSABk>Fi1q`*g z&d&m=TcN2#UCaFQ;V5P8HKnVpJ5kMFfCmDgw*3653gZdzGQqAauB)pn*YuR(g!Lai z-T)Jr$SXd_eb?N9sH?xdP>+*vN~NzlQ$<(Qs|7Y^73fn&-bnW8HwWRN|1q`$q&AuXoV*85xpCf)%O&jD^9=p^53EQgcD7qr}%e0 ze0}i1ozEfey^azqU7xu`cUOF||Mg)D8i}&M_8khE?(Xi?UgS*Z8SeTkh--!_p2I;} zn+f+*Y;)n%i2V?yEvu%bbzdUeP@T-9jqTO>>t8o6?5&+;holQfmP)BF+;5Hj*s)_& zw6ri*s~_^5tEc+NF0ZE-Gio3Jm2T$&H-#24Li#Iq6MTK)4bgZ}Ro~jQiwrj{P2vj-xMJz>X0EDA00u{&5%;UXVLWI4%y{vq_5g~{%^OLnEz5Er-w1ua2W2{{75GWLM`i`atq z^rPm}ya$%6_^?CyXv`)$-T`TC(baV*?oQ2ZRJA=KLRx8fJEFpLjx9eSOd#Wi(%L4v zLCI{R)c=GQU0rIB1-XAbr2$;{#*Sl=j~lKWIC~`NK2%;RrH>P?v-4SgiII_%E!&SX z-0p;+0MEaWiH|-JhLO$jgvUeoC>77wyGu6`dnRE2p{1pPa6(vHNXje;!@e!Q<9n@sSZZof(a;k_U&LrL`B8zWbOh(0T5QCPGAxL ziqaZ;5o63oNBURKZ(fi6xH}#w{K0XOBavHnHz&bL=Aiuk{rhx}*Hkl_7X%8w^)^3E zV6#5^r3}?z%GyGdDNFzPN8XY!2EMu^K6*s+pDgn`CHyT48aQ8i`rl?1W6^{MD*cjil*{52Q^=uWdvA-p!gQJ0TGo#5NMCNW|e=iml>;6Xt>_-fNC_-iPd|O zGPp{Wb|{3f1^&r8kUTzK$?m0T%E;gYAW*U4+0GKD0T%!KRNP>^imvXjs8~^gvJmhB z1sdz~Nbqvw-8~1n`S`?U|B|e;{XWexQYl0V92=inTibPn1=PDzi0*_fBOn`;14Mlw zd!j|7al>BL)m7nmbXPwW+{MyAHd-}PIkNV7p&={dEB~n$5Hk0%H~HcY{o5g|VQ2is zB8x&v`PS01YyT_32t)4V9%OMuPrdoK8pr@KlI40YI~scoQUCq>H};q{4UBY?+2IE1 z*?LiS3;r{@)JaXSXkuvg)#`?)r$fSe>o|0JxFJCW$sd$GsDjxLHgF0;)6=^gY7VcZ zZO$bT*H0FBYUhI}Ql1JC&O-6$+1V_u)+0?x+F}etwB<`XAW1)m0DH);Bm(5Zq>7SE zf{_gN4A7!x%T+P)T3vOKxaT=J5q`@tetnSoiLM>Ul|E-C<2(!g=6 zVo@z*Z|xZz+S;p;P{h6|74|>CcH2F*3Rxi=8v!hnXw>LkDDTU+a}$io5Qt$2`%*9; z`I^C~tywG#O)a{k<06_Gq>K#1!bs#Mn0)4RGK(iAA;33va30-8uNJUZSGS={_}g`d z8Q2x_I#dK<$nu@%B|sUN6Uorf5J_s2zqqUT0Ww9cv~#XcBVTXTZ22m-ALa+-nCR0w zdU`eS$E;swW^z*Pg+BzJ%>=cWczPmL@2~1ptiNd*iTXGxu%%sr|94R%)EQN6?V3^N zzpa<-X8J<~DZ^_($Dv6c5X}#<{&HHNkb&ZGetJ4`0wHo6GvS1cFFpVLoOq&^ZTY$@ zHZ~#wgF{0)1_nHpFVq9}1A>XfRSnFzLoIt__YtN*!IGxBzU2xw&Z^t{jH}Wim$blWJ;v+`3(8okkju z$jLD;J(qcc9;Xya6A>MKgr7Rnq*{m+z$K63^45Qc70B|vZTA;)-|1}t1s?D=kbhtl zx4((3l{8aRkh6V7&iTfl9C}Bw(?B<`U%&1?lIR8x5nJ66e-d#T)8H)_$||-D*R(p@d*jbe`Cs}O^X$90;$q!4wg^(f zK5&;N`>f9$a~o&x?xw=Uiwts0AIag}!55JsRC|JQ!VRHPqK|mclbjN*mdT;FHAnv* zPSG^hR_F3%I1GFSZ||Z?Kg2_TcOZ9c_^wM^;CR;&;-17A@B1d}5hkpO{E`6f(Amhc ztH1#YX}0@}DQrAPy*YWI_hCX}f>Kb+aZDWTQL{vF0DE>4P{vG<_KJ>I) zuoxIwPQ&%jGBWNp`7Y!Rj*KLX*1kCN;GTuE(TUl+pH*vU!~bhA!n{#PwcI(w6H)=M z1jLG7Xa0JA=jh}lQ_h$=$JPV0N1{(Vz%qbLQ-$yN{rfjkAC5rp=L`&LQSY#>$Qb2L zsOqrNPjH8ruH=Q~RrjB!>c>XFa#HhN$cFGoXt%^^!x2O&cbC48T_>!5>!l%^O^%`n z%@x_ym-c#{@JfdtO zaf}fn&aSS1)8y8J!{s(&?Zq`%D1McGfx%>$fLn4@k;!0>i`cO7OI&f^xDF)qIxJW^L zo7cgWN8(VQI+w^}GA_1 zF*des{Bqo`hcF5Vskfx0t6o5|OY|TL&ldqH!P+6^C2bNymxc*6v#~mbMADw^TS3*| zI**=(Mx!ClC~K{^%g{@m>Dxtzc^0;kqV7o(^3!x-${y$sKl7{(Hc8Kt=2|upO$lyp z4xT-vWcU7HuIJeW-zemD0IUmP>uJnpC3CO0h6EOCi0B0kIW!HI|8Byih%ydVLn-)R~dFkdRdVc1k~X_$rqG{pq{xshMNn z+uCwnCoU4O3p9p@oBJ2mKMk-Qc8ycryB`d!V=05zD{b?`^{>45SzUnV^bz=&$V6of zhSCZS&$nGp%gJ&0S?3~5REAOKvmlA-y07(1+=Y>_i!t^jvV%D|P6#3pXZPE4waQ6p zc8kcj2V+Qgn5yl63ipCIK=4_vTU!%%4;7~k$b{VV)tRF=SZ8nmG}XQY(`tNl-AlybdCXl{T0Tn7ZlS-{85 z!lBsw&+>v?QrYqGPmz`wLp7qa>FFBEoP`{B1B*X1tiW0%)GTyTfUsYnk0uB_-h)K^ zRgzQ5fAFHb&56U4Wyg7W_Z4?HcD%Fp_;qWSMqQx^9RcRAf=EGMA+?SZpplR?z>y%V zJiv9RH?cw{Tk*levfc;M?Y@!^CZj< zh-`$K3)do5BkneOn(Vk;(s+lT53-WsfU+hcYZ6jJif z9ao3~Bw)q8P^kX;Z8UB6N%bz1A=6A)mx^q+T1GkGiB(iomU7BwB#!qx*crhNr+j{!*0y1!V;&eG@d zAp$q!uK(U9jz>&c6jgzuSQeV_67%MOA0g*l$TDuw_FFjz)B&Fe329Ts+fhNW^CB%N zgN8v`XP$G)+5aLToCUDgl-I9AQIH7Hl=$TY!z~g0rob98r&2{qdErmrNqt(SqJ@(z z9{>i#p;<&nM{6V;54(5op2^~J0TnRQQ!9o5li0oQzz(4dfBpIMGvpm46wmAGhJBXI z{kbq`FFq)lKgj271hm=sSfY|}a!_{RYjGjUfkWqjza?myusutBsq$^*2tNES^=^L! z01DN=o>~E$kyDoE9nU)KAq#JhM{|(AGv0VRBptAt9lq;t?>p|JnlK@+2_qNx6gR zRB6EN2Z~c6>}6MzeRD}H^;-Npz`Pl|V;{{K^ozIVb=(31sgSM-ooaKVcT)=f1S100 zm1k4cmHxqR7?Yr}NaTgr+)3sJ%K@i@ft#9Ct@>QkO`P#sJg0y4vHL+1%^>zJF2`W~ z4FG-HgjOrW ztykB*N7XO)w_7G1p$taa=?ID$lnDZr78MZC(e|BJAb+ zQj@=MVhDA3Z6Uwhyfwq1lHs+{`M>t!(l%`m?~&oT5e_lTQk%~AVF17cxCg_6Lh$10 zQ}+$LFdW#yePi6A#vTO({9h;8~HDM>D! z&_$Zbi@Wbb4@^QJ8$^zX?@OKhCwf6TOXw>LJS4pDhuDqpdVIBM?2q3ma&dZdDGEJC zJwfRu9;jgXn#ezKc!mfnpe{q_YjFDfLOA&9Zwj9`>FJfNZLs*umZ}AbXfg;QbS{#A z>El>&2R7J&DYE90STGvb;zW4Uzs9LPW8MDKbt47^*-pZFk96S6WK)cF%~8sO9*5YE z>r_tz8Z_!3{`YUpIav zM0K!h@H41>UyyClP!Z^R=KI1~2);@XF_klOz>2WrK!e#Lgiej^s|)^dTO|I?LrjTm zXJ2N!aXg`rSE(h>WPa)aDHLpe>CLrSB$*LU!pW_$$c1xXbtAv~lS`XX4Sqjr0Lk## zK-KM>P+!9PCKu1Wq9PtQoY}#mu}m{;P$GMQcGfh)9-%7ax3Olz-%`y+#7j-+0Q`}& zQ!e6;RR6%>Wx6=oP2dqcUm!|g^-H#e{{OVV>&mN#3f|~n{E@k+11k2H&OURu?J4uuSR?LWV`K?+)56CQ=B6mvAL;Qc>T~0ZB!_zlZeYabCB2 z$T3WHF9Uh?<8yEmV> zdv-LC+4nO_?9WoK6F9WeOTX@P!E*;*#CfY3%PMa~*;D4Yl&Q!jal$mMRh7d>2sE+1n)J7DTkYsddVy-9p^t1G%T4^nY0d$iw*17Ef z;ZL7Fy@^%{^xrzB-WaZWr47?EfR)P=c7AneC2`UoBHGe3+u^gpdrz2uiEzu5&y<&* zh6-zhp?c8GyX+=vdEkzNgZ9Wd5r&IaqGa`8Z4?v|1oL3?6P|oVhRyc{WFoUfXV?C2{6GY>`@s6-J zm-63&6~VNrh4)jqzR($mi$b3Hv4hTY@rSC)f&nI15bMeKR*CtknyJNem`y@FW*-e7 zON8o!(=Q16{CgiT&C#itV{S~}n>>v_0Fz;F0)WpRQ7GJaf1fa&eCsf&# z_aa97cF!09g|1GOH~snXnpl-k8h!x($7r600&ND#m?knb*lk^He~E8<>+d4WFJL5r zg@T2b_x0svWivSQCv@jtf5o@HZD8$?Mpb)ld zkKd-xeMX52n3PKi<;#bF$IKNqf(MVP49?L{eoi1kuOL(}VumFyt`;T?5!S>P*Wk$z z!2DZH|7m&gaZL0;<{797w*;~#78YF4SFxenbdR#6EgAoXvAg!);D8WW93nOchnbKb z2vZj#QUl*}k(F19j`M6Q{t+Fv3>c0A?U+`}&cx6fLNLhHh!D`RpuoL~|M$o#f)eWK z&Hk+uoW+q%K)4K(R%+_%W*}f5qT_N9&83w zrZYM^@`#uezTb0CKlUVZI$d_t`$&hgU;UD|V&+pqe zX5(0pnZ~^rSv1jnhV}_p!fL6leU_DFr~k!40MPB_!uGPW*`6MU0zSe^{`uB?AK(v| zkmLWNTCicQY#%mVVk>l|(-gh~MuNka^77^G{7y|UJwzb#-riT{U3f`DV_JG?ZPQUD z;k#HELXz0UP-j@Bq%dg*#I`FWS;%*7>A>r2tv6I6)m0R~A0&%|+;4{e5}_yW#o}J` zjy$UZa9rAVFlwDytEyFT)QCuCcrrKk0j?d-?9h*cPa^!Tk9-g!B{H5xE@K3zf&O8F zaE@hMPyjLi28c=&#B!L^?O1-~o+c-4iI@t|qClOC;Py%(%*&wXvLPa|Axfn4Anp$1 zIzfSFo8T?qt?ctvXH0qb@ia%|>qE_Y20k0Vnq+UHK#4&gU^o0SD|+33#}#FnOK=g3 zT_;Tc4%QMD12{2hC7ojpbx!wOYJzR@%a=i@k>QY*AzPu!GJdl*;tMdr$fmNl#PDfF zU0p8lt7!*oFh!K?mq9;3h<-HZ@ zv+~6P4|RC?NGpFqc^V>&a&%Ziq>{siCzM2lb06a;T0$@D3k8^7fq)4rg)A6SPbUtl z1ScYDXw=F`UX)3GG#)qz77>WT{SKFR&9|>PV-P8{+8ybqWhn8#Cnqg{cGeJTUAmJ$HSOg=JX2T)Xu1P4;6Y@8z{>%3 zL;d(C#vj4lDsQs0B~y|7Afkb&qy~kKNBlMy{hE-CBC0m_{;P9o@zz>?drZ%S*9<^v zK~)6AxPtyc989FTW&!WXn-6c$k9qnc^#tjjbmwY!_?af-EVuJ2hKKh5Q$qm!U zi5pz#w1XrP`-+nU!d^pmPfE0olq64x0$^+T`h*<;+2N%UT&B1ff(5Y{gfoe-Gm))# zL)@0NcsSg5j$P}qxIFZCjQ@T5q)fE8{_0Rc&*G87`_!F`+9->lM(_`?9**Nbg?;}< zB~3#%UU-dB4l$WIKf+!I2I3x!s>Dy5;m0T zuUc@7sdG_l3#~RS`TKbN>CnD8}N0|O|QsN9I zIARZo0Mf#}Uh6DhKLkMl3KNkzIcIE4Hs`mIa_s3nKrOlCQgXdRW#2ij2AB|;XCh~2 zWyJ$`lY{`KO+J%ONgaN#)4vo~DP)=dUlw4htd2;(p*e{XvK*KqDg1WWaF|EI_~OHg z{4_LvxNp%yw7E1d28i#5qf)#IX=63}Y)gx~CmH6&WNumjdt7XeWj@k5Q2LeSYk#hONnE!gFt`yuB7 zNe~VME)iB4pMLgDLQq1HX>sOJ@>g_p!LUl;yeHl}aL?5S?0uS;8|v;x>Tiws>;_qb z@vC4jlEhqwZ+TJ;lh-}M3PR!=aQ)M*ZM(1)8O1_!|C9C5!#&1xYVai#;By&h&Q91h z@DVUgT40!|W95JNM`@Oj(yR0i!RI1t5_^ZvD61UoDMCVo?_ypMQFMxX*Y6XR9s03| z`4=i=L*(W?(e%xF%X|p!AJAek_Y47$3Hhp0XsM&E>=PBDmUZN0k81AV&u8*g5}9vF(b%n6!F7D(u~1g5slzcj1^V0Y_QoS#Yxy)QRB zJZ&Snn=o?+0MM&#V}H%Qw#*EQeK&Dobe85M&9x z9&b1}8l8;TwC?K*4rnSw;u>{==-==;p1yeDj|Yo)2w*6{6vXud*Pa~3D+1Wb02(-L zD3Zh`&*C$+{(kL^4Q*)V7;^LkdmsukfD!>_-*?)g*wCi6SWS+VU6c~9jYeehcaf~k z@JhtVC9&Z4zdt;)VEa6N8fG9S1RsP=`^( z2t;&w`HP2@+2cb`6pLyNZH5*52@Tm- z-j)rD$Mv?iEwgQ1HWmE#{ZPBjBfOY{upbHc63#Tt6fUa0!gynb?B;S4BnhG=t3TrF z0|3WA%%GInL%l)6QaBp@wlfuTBN%8P_!{wQ0nGgX1h#Z_d3QOXNr9YW3hgH{$A}H0 z=RJ|}qLXmsK|F<()?CAf7Yv)FIkwv7gwvPD+w8g z7*#S`nHy#H(wMPp&AQwabbuXY4oOx_iGjvqTqQldpNQLq@Miu(Py%pi8nMD@EdB$S zO?WXu1#B-Xi~87)7}7B)cIAh#p%EvtqrA5Xa_0K8oyL6rhgBUzYe|TtD@y)fWCfGc zkG2_m@k15FetY2JDwPG;NI+0aVPpeaN4I+?h z>Xq*=buupHHyU%}MIZn(HeCe~L=(l({sXNv%x(jYuRc>4KSpG21qyx#+1-ov5`ir>2URD|u~$ds7$QDjixKVKCgUf$qZJW(r*f`iaoYEw9m z#bk(BEbM@U2H-cSK3A)P4lu7Wge1e8TE68Hwnre1(7UHYVRC6Ns$tL(zH#qp1n^M?WLY zLjIC|fectVu}b7!<&cOEnABGjljQrtfXq zD@NotJ&1p+>|>!LbzB)uJEyBVpHp*BIfc)-Bo^{&s9@RW%Nu`b14!vB3@j6?i2*&Z zVR1McDav7Qr8Lr7YOx93e?JQ(A#P~=8C`@awG*G>Hn|6fJaS~$K1Z+<;$z9i${0}< z2$BZ)!j#ms_$+cFl0$7>EiV^bG4I{GW(@vVGV#^1F@?{9f!u=+EacFy=(|+Od{#&2 zF}iXTI~lPqjkAEgTxq6WTp{faaJ2yu4IF`!XXVFaNhMo)$5|5@YDN2eAHq;xOgMYC=!NASbpq^4m-Y zrGp9&u_Ye=dOO5f^2V;+IQo#MqENjFBN9&dQjOo53c6}&%|OnbSEgF?YLU)=;DO-H zMNK2Qot87@2x*~-H{y2!D?#Y=>Sdh~+Fsf;JzCWGI;-pp2v9w^g%aotVLTVC{CTaG z)H%uD~HejZSUin!)daD=JKS2M7%+ z9tLj7?Mh~L;!wTiqV@eQEW<=jyozdEf+E!lNAxOB`;#+2y0T;oiKU$gwW|sDp z$9RiqMqzWq?=pZd1=EEHivVcArX+5h$Rojgt`jJ}w@;ve1SlY@X|k^AOeDu>6vb>u zk)`Hrvlb#s1Uo?o(9gcIr6>~(f=5tr1l*7atpKF3Q2cJ2UE)Chq&S>GC-Cn@C}Nub zwPx7m_Nt^`t%HV`!~F% z1ZWUvwF@H=FsvE#ZM|hX8xnL}>*(7Y=Pjed)eMLiv`%tmsO-vw`(^Y^j$aR!Nt zL6DL#dM%%L1Q0g)0^*MtTRmlt^5nHj2VIsYby21GICZ%CC%zmqs*LOvKN; zI)8M^WmfLK{@G)mbJs92jr=|<^Z~V_H;&V9+(j0 z!3^7~N8-!m{d|t}0hnJxHrx^O_gEfGRuefZQoP@;Nm4LkzxY^2F7e;-@^Ix8qC65T z(zGh*wvD>LBRTmQx7KRe990}}ysgVrPqHx|#T6hEO&c2w6E%7eMwB@y`z~(`UB0R| zrG1*}IkLKqSMUzJse^1s&o4wzQZwVN4MDyh3Ow#L)v9ID018R0ECm8^u9asp)lcco;*yfy*kcyV)Q%L1;v-?*DceluLpVeY)cM% z1BJCy*eQG+m~aLYGeocKbR?^MwMdfa?*L-W_z>=M?Guu-2pTKYuEONY{M*jJNGm z(h(x9G!Bbs^5u-GosLRcgI>BcZYihE7IB4W7+BNp>Md4EBuWHv^>h*DP_N=sz&w4^4)h{r+jCv=uwKcb2uVy86UZfkk) z{NNGEl>cPVw}+IQ^>^Kohx!v$OA)4wPzW)}hxju=h`!X-5hE5Lcz6>IMR?83vNv?S zWUKg6v&`Mbh~{?h2|8bN%~^?pZ7ZEHy(bIRAxS0&AEg?<615?`fr$v(;)v%%l_cLm zgDux0HbEvERr~W+1-c4~Vr*(H3JUS&JiPem3^y^x^j~d7Gq!Ha93nD1D6L)gmHQt% zHA;{m)?Kz(I;8_>0H$)C*$KYs8i3F)MClB@Cf+@E?XV&tuAjh4|BtvgkLtPo+P^b} zWKI$yLI{;ip$wHN5h5gwWF{I^%2+BIge2*zD2h@Esf0>Gk|Y_522n_nko0?;B{U=ULBM&mZ^wby1(sdpOU1?tSdzICh@f-*21Ir9w>G@pwv=%(=@BnjxU-se_8% zZ5N`Q$J4$+vThILiY8G#NSb2&VFKj)an;V(ZQ6CIJHHZZ0oKLPIX*&~`Xax9o&2e- z{(*NN->6DE&yJ&Pm=$W^oB32F|A}YR;E%>Z`jQ?Q`Ei!n7ISCr3NLFq9z3(hK*Oyb zSXCl+%$|19(_r-V2!r!Weyr!=@vKQ4*gC{-%u=X4iHOh)TOI#G^oP%e2MZPcKQU`c z9qFoLnkj}1_x%*f=U{2VFzQF^KuN)utiDx(^%(sCPwaO7y%&|X!kun-0I(JwUz}S! zJ!i|P*&f|`N=u4bGf3YZBf(bp!|lkbB~r>P2KTrUw`gMUc|03+4WYsCHFUr~fAY+D zcO_9WImrLZ3EQj*d2dF{aTz}3L$8L7nd<|Cb9;vF>*PgORaD?X8XJzi3F=ZNl(w1Z z*TG2sbw2Ajp!wETs<^&uUHn-SNLxvB-U=HDgud(PL$%_ou!bxJZ5%ONE6gvDU7$8% z4~Aqg-$cGH5~~7@-PA#o>9e(QZ2Eki-YCky;;O6ba^;1X6w#7pt(#&A4uX9m=J>v6 z1EA$ie|stMvFP9L4qY)^t6hF+MbnG2vxzDXyG0rqCDjJ>E|yO8X5`b;S(&QRDi6bBVjgb&lNGro>rV$|!qf}u z2sG22gN^evm+qRMJn-~~XSN=aIt{mIQcn!7q^#jJLU5q0CDKO4TE@$9=0sv_fUygN z0r>epSBLZib{ZZK78xmtnvf-d9vR+0Vt?EMvG71EPpN=ftf|3dY+#5&-#bu#MeL_{ z!x<;&$d3;7HML34(;t4o9A}Kq(AnSuxo#U`_A138Zp~cf5Cd2wpUt8Z5_-|%C4nVt zzpJs3z87wX0G<$63`Y9@Fx*baW&MHisc zfLuvoIRv;XEI5Y;U>5mu>R|70TWxG+6lgeW|6`uv?r#wFxhYFyA(aQLN4htBjA-o& z0~iV&fesy!pHMwJBZDPMC?fNvHWq*pRLypRKnEG_2#%ukqF73FtnRSV9tN#Hn*+Z2 zS-*g0Nv=2sclX?g7FIU2!V5G!v&)6gj<^As zI*+?Oe`~Y%O-adPXq|#@0WT?T&*x2mk*V7M9Jar<_#u?|sk`ajvdwrE#1NQmE zyA2*6^hmAuo6HG)_C?*WkB(Njo_GpYr4ZaxOTyhO&TjK*eynSHX-R{z%h<98-HEea z&rS~A&^UO(>p^8&51L=oph4BfKr^sox}NwNhM~5 z(MGLnIWAo*u4-3;SDZ(+kjlN8;pq-yd$V+SEO7zOyB0vsg9$|@OJ(}`Z=Sem{V1;; zTd4+H#bb1@NUQ{7mF6Id#o?lS2zcaaY;iP^B)IkUwn9as;!S@5Un|Z__$U0+&71N#B2N0lacAFdcg6jlZ$G#w zsO{5#y{&BI#2%g4E&?{<$Jr$`|3K(Gbx_y9u^=m>t18!0r{90=2Br6H+_<&;&yJ)vh$-;+$d<0$ z71)2k4;`12*{e0jst=zY5;NF+`OS`rWlo1zX_htE=FQo4DtVt#qq*PfyupnFo_^Gz zj@~>`v=!{_S%yMfkK@LV#RWWi7-p0OGP?2OdqSKOvK&DgvDe!~Ht=INAlQz3{nigi z95YK0l86 zsCZ^iS0Uzl|2Sk*_kC9C{hofj8dBYNw(i+n;2Ce z2(@5V?b0MdaEN3ZWO?bGJ9jQqA^m$(vjAD#R!dbiHhq5A-d~S8&|Yx@iM;RB;K&^I z9xNF5ft|4boAjM;hXc*Ydsys_a#q*5f(=tUT~XNVJ>u!gGjl(C`D!j-Yd3fLnwZ{c3Kln%1C2YhnUg+#$m(MY zq}4o&Sm$8W>MU1@rj#A5(K-kL0rStF8o%CW`kx`J9igog=0y~NB0CbP5^xxgZ*2M> zTcJuP7m$Fl@(!B+d!eWum-@A>Fg-yi=RNl3VR%jnRyn;8M=T&KrHeTMXiQjzb@LEs z<=Wp*1AoFJT7dD7kcb0$3Rbg4^X~@Xm2=f2Um)8<{NGZtV|RR%2ST1_gaB9A3c#JX z;_*~UciLbXi?IhzJ$enXnzvA1_af*H;llF0W}ld{4-!CRgdT>sEmSvx5$M|N z6r22Bo&x>NxRqV%4qmAFt##!kA#1)Iy#Bhh&m#HcxPYlT$5MI?p49xNz&7p6uxGxF-_K6{^t7OP zh-Tj%e;_B=@&qxMxB&(_8F{*I&XeyIIuGtwoeem3-LEMz!0^b*_qha$=i3D z-Yf}KDp)L=U1MQ6_q>j-jlcydQdr$LV}(ElSaUM1E{t0^Wjh-g(Yvwu^RtePI`-y= zAr#*PN5MM~n~$uKa(Mmv!MEy;Zu?3hNmCcrK{uABIPvHLlDANEn&1^kKy`WQ`?j~* z!UEGFA+=)$$W(q z?sn3)t>`3pl!$@^Rfi=xhDGx>iHO@WG#MoxJ5dV(fdfUE2=s95LbjgRas(ji<}{rA78S`+6taovS>;4Z{ zZ4cDEzBzPnbK;lAg!^~x237a^=9O9FdH&AyH&qs=^ipq~`nGhgyZIH1EUEu|{6dOu zgHHT~3-&EyQ1fAtIbi~Csc=#!^17UYy>HZo@v_s8R{6-O?H9uTZLZ&fg~Wko{e`@7 zS5Bq<5%3Adi!Usz%N95hsw<|4|8Si4BlEIFY8X#>)Y+KWn_dLYmk-bR1lrf;I!`t= z8$4j!f?omiE)J@dn|}18L-$<<-%}?nEGO}>f=D=Gm+P@-0k0p&M)~B`G?08dKJrqJ zpEha5m-fc*9E*>iy*21f?OL^QqZ*s)4O7&sZn=jm>()K7Iz60&5g}{pdKKRMN$Z}S z+W+M3X>;a5bCDjXd7MT!tJeEgu5$J5wmt^uTyVp{&5m4U%Be`t(%SW$_^^sDu08n% zJV4#$=XTL4!oI~u`)JgXk!YB^B($GrD1-O+^pTfeRQOZ|o;cCx($B?i3bo~JgL@D2 z=y%R#iof&a^jqTx^|?}2Gr(zqet4fYOSHDY>A+X%7!TrD?m9|ZL8S!6Vg=8^GlG<7 z9G)mYj{UV*`h8!PXinqGWdPHL2J^A$L*>o#!*!c&4ULS3$w(oC+*Z5PuYW_|vIT>m zhHSl_a-m4od(+ZSJ(K7n%=vX;V`THi^XEka(`dz-LmN(WqzFnY)}2VD{_V*o|EYi2 zWL-CFt@7iNkpf5Ajj1Pcn4@VHW;6mRQywTdS8rms9~ks!FWXXKeM#T_r}UNFXDfk` z{#A@N@p`mLYXaVFoK7jGxwPI8DzX&<6B5)(NY}?58+}dRck%aqhc4JpA|Y(bv#{Ue zX5Af+$8=0a1rWZ~x9fXZ+mY?nE?$g?h`3NRZox67uDh~>FP-i_RPTu3n|1S`co<4S z*UbY(FM0g9lyUDnpUIjux!|aAm)M${Yd*|*r1>T8(;Xg}uz-LIe!XhBD5->&l* zfh}8LmkGY0aN2=a+PM4KL>fkVN9D5s(@&9NV4xu*D$AONB}2Q||Bj={?iJ!}q)W zt#VsOjFg~feMoFyz(aZ38vxG_1JnOl?^-kNREIX;db#0brb~s2A{R5KD0o_vWrNU68sjss3*6Wh2 zD{;P;mA%q*RR?^2foJNw{d@c@CeJnL&8>a>sq#=9w?_y!GX`&8vgNwiN7MzyuFaUns>PC%6UtWFJhbOMVI|mOx?4_R1kc}KGF1gt@ zwfB@k>7{F*eH_QawSbEVC5G1S*O-~z#^Pu-vy@+Hvq{j%lH_Ay0)S9Th5duY^ICoj>_**sAg z$*e!KLz3>n*7Rjz)4(d48f&W{;go}@P7!pz1QpbkL6f$b&UBxxb4}-QoS)J>O|`Vo z{iQR|85FNRxnM*TozAdp*G>unl?mArupD(c02+j<^AQ6nYU!$!NVx#7yL{fXR-h3b zid)+LU4Df!O0zcmq|!WjeLk+2=Ziyf=L42r^-T{yXs0ZbFtw1{i7c$O=!TH6ZQQ)i zn?NH(KYQBm4-w-1jdT0;kcZds@agm3!=9^GIj=C{S=`Z^Hp3oTil&-we`9DjY^I;S zEkC=b+&pzbbE5g(_9ad=W2mK?&PN{Da9Z@+uzgvph>KEFY0FvQ<3I%S0*@}RyD6W~9@aggc{1Pzur#eQq^d`!=z}skqMR^eX z$XDDsX`T?23Op72pT^4teRSiqn<`YS#fMr7^x#_Q#GBI!``DSOVX1HynFSQBI|zjK zrhnFi0M41b=jLnflOsovRE2{02AzExH|Kcr^lb4oJ+k3+haknxD`hF;OAhL_Hb91q zKa*o+WtQKC=U{v?OMpB)u?2z%|5&p|1r_iC79{>}mQY5-bNtwC+evFC<;YEU(umxU z09`=>IB+%%CyqUMIyk+AG(KL_I$I>iO1-wYmw4Pdk7^d2Ut;?qxo;}~cnDTZy)YdX z(m~Pb6g+&ob|S9zSz)p-uPQ?$SF68N&)!PwF)NPJY8WYmM2z^@+PCVd#!=D9>5wCb z`~3cAmjhWIn{^s0XI*YP-~%7OH=-~PTertN9XJ8E0Huag<5FDc>#lXUnjH7B&QVTt zmoi9glH1t#<`qrr3N*UXN-j$0laBrNu(9kozw>8{@q=Se$JU_042W$k86;)&&wQOH z>SIRCJN7JHzkog&%=yhbox^MbgWo?WGT3U7@EGC_6m9@UxZK6*ZrbkVGJhm{@6!#- zn@vnvHhg)C9Wq> zv@#4vojA5ncbrVa;<=h3*EUT&yym@5`(*kM;KG7^Kk4=McC3L=$5XSOdC5_Z(p0b} zi9z0k8x|-#3CS(T)oDZQ?Z#yO>DIafitNZFtl#`&s{HcyEE#16|CpEGJ4&g`+ZF0_ z2~uUqqxsle@^7Jb*RWHekt(o2@&%l6z~l(JE8JzmclLdCBjwE(tu>u#T`CQHsIgEw zgWmJ3&jBshU~}Be(IF?O;v|5SpJ@Pj6G$ zUC|Sy%-vi-)39mV-4@a-TDTW3U(Q{DfiAE^v?+)nB zi>hqxvw*vpC1W?SE&DjD?1+w;p>)8Hj^TV2P{Af+PLeZ%1MsInySX>hm@LSTBq*qx z=G_`s`0SYwzHy!kheDD{CAz<$ov>?pc8I;l)@A_T>d9{QSDN>r9p)x>Yb0gU788n< zrR)T!3Cc@}%eRHXj0O-^_!4ozil$s%_8q?d`iK=W9l!!xJv7|A589e{{eG267wm8} zLyS)vU0Eg14nM%OuTzm5x?d*@a~95&b`WIPoMC~geEMIqA5qMTc!IS@GHyowrrK{# zH?WauMrRGPP>rB}uv4tZPUbA2V3nYDS5p>R5KN6GFg@qd-)N39P#U$B8y z%ClHEMyz=MaAo_I+g4_mRdjuk?UEmVwmL`(AS8LsO+M`GVbA&l=Y2meX!qO8v%8Dx zv^4K23enLIo}J+zwZS-;z|wMkm#yS3v;0u6&+8zRKxVXvfALSJb9}gz?&hD+A+1l8 zoBHj`+Cc=F6?)g>@W953Q=#Hg^MwJ7;jZqd+8vTsxe1hpIRP!<>Ts#CnbCfD&FoGj zJ#2`ef21Dt$!NQxYiYuznN2;E+j$QdGW08ne)_b(Kg)HK+*cOa{c&yiC(V#`T92bF z`9H3vn`GQ6C0rc%hjq1V4?vuM`sQMId z*B`L@+y^fF+~1+<-A7D~%54SqTm5BONdmU&)fv%Lb3nHxlB4<=@1lW_1%0NNpwM0W zkE$^6`-xsjNI3Z9-Dc)1Wd-5y)wNvHsRN}By|})u)WemQ-Ti-N9aFj(8WcH?`zqY} z6hkH$*oovBLRSMiDom##`#$=o-@TJ*%NP}hrxUzLp+jbFK;?rIVvkdHYc*V4gaxs% z?*vf&{ldVsy<+4B*AEU^d1Nc1`yiU0!hwcVOiZQ`q@r54mZl%I4|B7*)B4%VoM_@r zkChTj2@y5+R-a~Tk6&1zaC0nqIEfB*yE+Hp%mVj^M|8VAOn|sTEiAZo!UIt#DYk>M zscw9xESzhEp(Zo5g7n`Hjaa01$rj5r^`J){?>Ds$%lkv)TNtnk@o-;LN8zBl{PL0iGEuh5u!x>}2+>ut17fy2De{LJ|(1PSlB>JL_MNl~gA!Dk;*gD75 z^hIgP_qjSARQtB@fP{P)5hYO?kW`g4%{g*AR^OdZ^z43U)@%C6e=@ZHwBwP{B!U4r z%g!!opc0USd$)mJuRgyQbUi9wh+%^M^y$>xV!S`6#?I&BnmIk7ED%+GcumVtg&IMB zfp+|x5vbre{d#!<7rqvklo+g8^QLpx1=u;#1QnC1dURs@i@^ng0?yKI`jUe8)F?q~j&p0? z)Fj8>!2s&xo4r&2-Ymo8GsoA>VTt+CWV?D5#U+UqDXR0w4NAT4)HJ;8?v$t9GEFq9 z;6VwPOt8EFw1nd!9-Iex1`zJ_3N(N`R>oU&Kx&}!b?T?Jgq8j!Cbo&raA z?gMR_t%Y${nk|tV#Me9eu{*`v3`V0tf);LzNQlIVfvsK+5``9DQBfM7#85nCy38%? zPyPbi1e59r7A;YH>o${RgbXWPmmaWBcT#jT@=rl&a)e5? zH+_qQsP-tW{(3AMWj8A>2}Zq)?#u!W(Jd9AY)-pxIQ>J?jEZ6`ekar3z4^ja1GJ3`3v6s`%xHtP`Uifuhumq~s*`$0Bz>Vl%i1!S;G~B;EL_cLoR7zY z3UvFVb@6N^T4Y}a(ch;my<%wo$Z@Qoh{uCdQK)p)~+?$VMS04$c5-jkO@=9(%cs_)z*ZJfl@>QvjYoT(bq-7 z;MVBimW`|mkb=@iJZ~}gLYNi{sVvDmoHj;W{+bWtsjC7#63i<7a@YOBC3_}561;11 zNF1JBWXJX2UzaF&8)AruXjX_fL-Ugwmp%vbaV!ty;JiRkyP5WMoI^>;#rcx5)46Fu zw&ib6+h{zXu`mk}2P6$}O#OmCFZ$1ZIDQQ^SM@OU6qtnAG!Q|3U zaW<|!?{h|kwim-FC^c!UIlSKC@#b&45m%3WU8dO1TY-Wk@YeTX^cfs!2@)b5SZ{*J z2@$92;8p@ZDo;wPw;#GTU0k%sOIU_*AF-+R^PO!O7&tO=0~7s{(ddZdP$2NM9JH1X z4|vCC{HO`7?iojD594dtq*UpHemCLsLc#pW`T~{7RAH?f8{3VcAF(yl*B$S;U~wDu zDuFemU#oIDxiw8j1&b2_dCkD4yuK#e+mIbWpfp^)+F4&ICVJFK z>-;BAZgcNWE(_yVpPd^;wmh$yS)nhBMTWxyZK)sEVzKDyrOTJkM@HT&IcBbEJ=|vw z@2y9}Xq`Kk02^LSVg!lt$%gnzK6nbz@TRzH>$Z9&a{m40-7H<6ZNlEXwaqL8mpiBA zANIp8lkjub9sO!zo2I3__RyI(c2ct+aQW8cN?9SL(+qhpTzf?8G$7}SLv2aG%^fgk z_O$c5bkNwsOcPdp6i<7W-(m{jckCqP@=6qoQATF_j{9_N@BII~NG{#H6 zf>ERJw1X3{{q82eCa3Z%#8i%?OSeZ-K>21?G?55BC_3|@yEty5M?uYDKVYo{$B`-% z?>!G1yYfd9@MYX@B@LZp-wf!!Bd;4!qgBtb=&g>n>TYF+bLsxmm$435)f`OF%IOPP zkV@IFWCmS7b*sbEI}Qy+IK!pwo*pTnw{y{>nB-V8TF2m|_7vXcd(i_xm@IaX*hwOb z!C?T3Gi3sugw(tUL@2@YCRP*ySO{TAAC1|kRo%*Vph9G8HZ&qoGF!cn^Y*}kgz1s`?5Gjam~kED8DVZ5IYm1(yE z4I$}3ht6dSkP3POLNLmfd>@l8{_#Ds$5hmbJ82gmY`1b{kV#2tmwK;CQWfFfgLi(N z5W-+aLJw?8`H}IwPvQAFWAI|XV=pa6m%#AIc;~kdR_+q*p23HAPg5v<{$sziiY>=2 zoo)h0M-EH{KrN9?A|CM;j%EOlhqk$t_cz&@J7Kr>q=f}AD(v%Rc^?YHOXb!^!G$hd+ACuk#0dSp(*D@=$$ z05H@@BgH`t^6)w5%&!XjlrHKUmK{^taHZ$Y(G^muCH}k`sAf?GDyy#@={ZzyJ8_gj zCc@5uW`xLhMB2;2mKqu?5PJXAK`UB+Jtg>oxu5v;Nwi$YKc0Mm1KUSDSAnhn$N6If z@-kq#;N16nM!#W&l2q5IiI+UIm9qUqw)@i@@d>Tg0FGnRzAq85lGeANJZJ^QP_fwc zBzP!W03UaU)DCu-0KfPFhOU)=yi8eD-Jzmmu~8Ns6H6=*Jx4*C{I6PV6W+xmswNs$a?Xlj#qlk*BKq;Q)JuW_0wwQS|dVVj-#5o;>YMVr z5C!(J-_-kYGXuK4vQwcE$cW?(t4chERpgGDckK9}@a}%IpQH`__e`R>FZv=pHgYC* zFzv@~h)w3rj9$AgaV5mR5?_{z#*5Y+tb}{1`3H(kT`tQ!%d3y*;3K;{N!u0@y)nu; zf|I~KhxX`H(Qt+`QNeda+`o8(#=@aY%_1;HR^u-nyH{v8&nfiKoG2hUo9Z7#?(P*!s#>Ti6ViNs2|ceHkpZS&Ms4Bj2A7bF>AL&GEWq+35G{Vn@}A`sr_dHpvNot_O19g_$ybA z3H;gUBdwyfAKD%L^E12O2mL6$7`@7GiY#le`4o3};Di`6uNW9C!n78%PT}>1--t)9 zHVc(?*dhUm9v_j|C-lgf6ZL9diWek4eEKvkGL{nS*fVYHqCW1jz`Ek-O3fYWSL6os zen_|vX>}#JUwSrNnZ1?Vc_mBDVlV0C!vv{4;&6_d7H{noFMX)HhWMRjMkiWN3)HBb2?Y(4~zgk?WIA!eH zRy2RhXU7HSf;v|=Kg%YN+>CvAtRziiyHm_eg944$TOGm{m0Nlyl@@Q7%}}+fdeW{y zqtfn|9Qy{+_{88b-!ICHd6?kX(%iT*!K!n-vA$&AJ#&lfpa-W$rYUu?>e_?j4Ek-K z-x^vcBR8bd+7@vZvTO)x%!j3;G$9g-(Fh#z+w*n%zT1B0_uAPH6FzxvFH7&5F~d-% z$!-6J)2Qm`->nmD5`mJFhb}fMRA`i4bS!eikDo7-v-{X4x<+mYkFNHZKgq%E@@q>^ z{erp#kFp7wP1S$@W=)}V3%x#g6|}4j?G`HPCBQ)4z+6u`Qld8oMan^U$qqi?T`Sz{g=1% z>;a`33pEx30b6R-e8Hv$wL#;lpJ%Xx-_tC)&rz7+EQD|bVnA?HannAy0 ziqn=(E2+y2-Ptm=(L}#Mr>MbFGw*fL$B%N!ZnFFKIj*VGv^b@DJ))P}MASd^{oL!q z${o6&rOi)mOGN?&mw>vkfSMV)*^5^a^aMaOVADQKK(9;!Oz36DkZ5wJ>4h4(XO+L$ zCjZNCBN^lrsL=$S$z5kCll9jv-Y`6`svQ#u27as6FSy$Bs_I)K@`{Q#UTd0^|KQZ}Y22~HZ+C`v@+o@p<9o%^X3tqJ8xo2y zHNPxt+!?Au1=oBosA*S0ni|7Kg>nomiS%Oc8yh5&&Zl%VdThVi^Qp$?Q&A3eH^cpV zl;?Ew5c3;DLnm*`m+0@#^dS2KLroW?Mj5NDTdQ4{NqduhMs}NO8uw;9j*clkt4m(= zcb};&!&nH=i|rbRqH7qN`a~h3&(Ramc7t<8F~S)xc&x>V9a~4-Q97g?-;T+;j1+S| zTG4^JFrNi~LL9Ye6Mzz^2VD*Q0w?m#l*2V=g3Bf>#5;MmhI~?^9)J!se~w@wWzyzJ zwF)|}#5n+BkgOCIT07`dbCcXO-Q@b{Dv$=jq;EyPnPV?#U4pLeqD4R?k&z&S14)q( z+gda)R^s-=HU-Ay<(S6xZJmUhRD0fW+Sc?yx2R>{TUb`J0qouLD8cTA!shK1doOI7uKd6>4Sx zFFXo>^SLzc3OV8P&MF!T|0VoLs^bSl?LOI>3j ziG%MiSG(zVwdK{b z;{Mut`4oTL6}~|p4K6Nyq@46N2Eo2H!*T;4Tvzu{bJlbSbnN2!=R5HwmMZe5{x@Zg z&eRNvM7_>OD)ow=+P1DSY;h;J^xD||_2l6Vyh@P?5rV`KjqA&zjz0F1L<)9skV`|X+>@b*3w2CdBN9WrdGzN%dKlt>DBDFy2gAbvs_aPe%uk7+WR_f*xgAX0{ z4K-p|d6#{NaECprYi@4Fe~E?9n~;$1*6rBYvp!iC=G(XX9X!|`Mep!{&Jp>CVH{N2 zs%dB(yko{tl-bgsGBPq|J*sJNQdGUPd^FUC%uF53*C#uC89-aZch4tFhHO|sto`<6 zW0IV-B=(3GruQSmo;z<|&n^9W)OwGayJcd3CP*LE8(I>>S1HbT_T!S2+6_%L!fR@^BBMxkym+ZGeA#dU51m(*^)^ph#p7vJV8LV;1s-jioOXk^_1F-9BP)K@Wq}Nf z0amkSqD2gQT-bNqgb63`HFk4L{~$-POR8T{VMQF{rY#2NJ$TS7a&Al^mfAubef+o# zI*1##m0n-Rck9}f84hZ4(iDMhjwEcp(00b>M^rcQ#wwzN1=|3M@!Yv{dpPgF=>F-; zm!mXy(AK|w`}Qk??M5=X#shf|Lx&E%n;Fy14~M9*Fhz5t#T~qhH#665h4lOCODm%a z{7;|u%CTY&8B>&n0g|B%WPP5pFef>4Mo629Yp3*wLw}TeQfM$guE;WJ)xVx47`Elx zXoN2{nrc=k-@8+LRe3SjwO(LL%;i*+F)E1FTzqlXXc+vR+B5CIV#e~0NBsSJn)fu6 z(S79A;56MdZbilFT<=lR_kKSubXxjG+H%Yi%RM#KdtovrlMp>jH`n{+j_mAF z7q-kv5(gJ_CSncRg_iO2^4K4#OWOgjPm#vVQ zt(EDv{?q!pQr|_+n^zY0yZN$YM8f2I-TzwByyJZI5ET~$WEvjhETonxPW11rt+lAm zd17Yv?#$k6)A_~gr+li)`i(OkqIFN{mQQ({QO(G|wdKhX66Te1tGDfS3@!8Y%{rX7 z&1IE91|TXwT~*i3|8AKqhNfuB?g?W$cMlJA9&Pl}KUAAVm=0=5$WH&h(L?)3hvn;o zHje1)sUvz+@3uQGo71B-KC?@g2Gi>1AD@a36xCQm2r|CBe1}^@)4%nhm-@Xbw@a(n zZ?1~yws>Rql;^br@{Eh_zi4BW8~F0hg#vj@GtS86i1p^CLB5_(7uQZI*;Ae#aV^oadPnMd z)rl>2`m)aWmth9ixAu#1V@Xn3+p>Zqi`HM7YMxNL=(%ZmXyc32BYk8nLrabO);R8# zf8}20=F>Oa(lajVXH)jl#fxV!5cO62qZPmLCYGJ^S#?=p){DNoK6PokEp4*p`o4Ak zwb4%&w`fgV*r3_)ccaPyr?lVOHqW^zS)BZ4>$^#VM$dZIrc+6Mkp8r&h^&WQX|qOu*XWe$+EHg>mF$IEVsK`u2yU|*>uYA z{`&X&%Z-Tk-qf{i(b8?Wx`~;nXbo!QmErkyYBk@}!9TNUZq)eR$#zOBkd#`?i+!LIzUyYv zFgt7a2Rd6e5R##iB)5yII5|VAOHOGYyW5KcRShQ13VZVE@y^A885jGjEi76pgQ7EV z+}ih}*Zz_c^o9?$wSBh^>67$1Gv&pzcHt#8bOLZ;T*H{M!+H_qFO*U)2TDpz-(KD` z7|PSH>J{k^M9)_I5Is!2tY!0*Y9}r+o*Q|vZr!}-o2Z$Pudj5wR5YVrC0=UG&1S>lZo+i{b`n zoO%T?*Q}7Wdik=Hx@)|3e&XR++Ejwy@>}80 z+v<^M*W32*7qCy>+h7jbPLKEPYXhY5a}P@m+x>4CuMCPQ^wNvCvoR?FH3GA0a#5ph zUDs2oKu%n}n8YdeBBBIk2nRB+#w;o-R3uYDfoeF_RPw+~;oF15fKYp_x_nX)6J5Rq zmkJ3AcHF@spB*|}UhWNrl0NV-*%YVkA=u0(rF$-qIgc>-sNU?|yOUo$?)OhrRMqq` zDrEkUV1-1R^KD1jCOReCLm-lIS}~eaKIzU&s}#HWO0|;e_hzXNvq@S-pO3?kd4xw_ zo^|@dG-y=l+^yKIb$o1=|DC?d7>ec9O|Fp~e5|6Yu{Fc(O z;C)xcmf@@DAs#IFXk}w_m>joduFEk1Z(_N zt6p9W+TxUEs;8$nd3U`MhY_yEj1cw6v9jQx;pRCUND|AeRyLTa7RvuH)s0AYlKus~ zehSN<41ynFMnOab3OrrfSqP|bg4{7&9}gZFHezj`bUr_KtGw9N?Hi2C#jJPyjEZjk zm>}3a*;+s1cU&J5I+nxC+;egFF*_#@DBGor7%}7aZ3*a(lg7u?%2#g)77kxG9P7d- zOE$qiS;sBnG%RAP7;huw3^+6K<6&61qjY8Vhhw7mSBGqdsKR5cCHP>l3nCzRhL289 z@MeQUf)Y^t($L>B!qP}IU6J5p$z-V>WSn5-Dbwdzu)B=xu~rk-mS3SQJ=d>=lvyXn z^Uy5Yj*ZHjPoLgsJR9>Qu$IV$+neZnkeYj{AZy$w&NXZ8!e7G9J+I7Chv1^Ok^_? z2H=|_q>~vDYcNMewhO`%q+U`x>aghR?MFe@&+M?W;T|XU;>qdkT<^i{-=)z z^&%994LbxP3CM9}u23bg3tHzt1qIEUHLESLCq(wS)Cnepi3h8yu0CY>BZ;zetNTna z-U-%5|F#V|zd>n;!G~R%Om&?6S*}Dg%MtA?@S8+v-iZAZqsp6xC)@dX_|1zvNN^Rx z&`q0SCJ&*DCftq#{QY+%COQ;wd{N;@OCB)J{by^I$?DY$G~}7~VYW)IE@t$g*h!Le z))-8AQ{B1brA_X-L^af!*E{IOET5U=8fnS#G4R|KKmE?|EjZVq&Kx*!K+K8=37LM` zKH$ue9G`zW?p6-6vMtN|dszGJq9MtB{F&}F>_W&{rwb?X`*by>01k4WYpxh%Tk(6h z_>Wuqp*CtcDef|TPzbB<;pd0=Q~Bi_S3CqR!yk{0WbPtA$sd=pi1{i07%5FCZT+Ku zKlWqmAA{7nxYj?aTk{R#Q#`F6_}!glyI}kp-_?U?cQ>UJW|e*`DL>!fd)R4tw)V!q z!;U%h=A*8wY@Ea4ixG#im-?8g_PU0dE!9CPwG|5gcDN-Egew&0%?|UM+c8#QRNofo zX{~UI%bUC*1G@F51@4hSaH`v_A>XHE`20<&S7(Pkxv?R;``^;E?_~~-1=0TK>dPsihC+WFLy~FZtMCfXjdqze^CO5Y4!;GAd6){3D!vEYkC2ZY62!y+&3-gQd3>Lg*h6YGnF;)Tf z5@K>(9XV5^yLR>GtlGG7lx9e`qk0GE>5YG9rh1sV*h-zpO^PYbB`j!QSQFk{)_sR&`&)e23zp0y2+zQj}|>R+9m8nNl8g!-CKw*K634|1&fqUJmhvn z)u00E>gu!58!^2ZbP}Sca?g0Ub|N20|MM`NVL9Dz7cA1w?R)EMhhqT&qTGfy3L%4` ziGtU~y|+5$2w|p`)xm$~771TpYIsT=MOh)fl%Q3Q@!g3YvhqjWnH4r(UHz_*HUKbS zj)M;g=b~~L25OFQi(uE1(&2s}IfIO*DE;pQ7BM~rAQr{N#Z=lH{iT0@TqjKu_%M-E z6po5VT$^jw7Cn2@Ja8RnrdE2@ANvKTjLvt>&qQVtN8HX#Ncg&VOQdi{0h$+?kC zZRZ_o$afWj^>}0H5CaA+^${Qc3-00iN9)Fe3{dlj6@2`O1o;u_y(cyrXQn%!#E}o2 zNZjea(ri>XzPanfg9!=A72M#J60idi&1k`06TV9HV@CYfy|?0C?oVx;)%1xxSN?PlAz zFFyBeZA-&iG`$@FVbJs9l~21*7%fb1d_9{Ajhzuw=#-JSYDmylf@iWdpohiwrDNKY zs#@oso)q5^-LD)6}Nc3>=;y8*B9QqH! z-Trc;yc^F(D2PDjdbjh|^wx6yGg6`qv)g*Zv2Jr9NQg0D^%0@~}DUa_j~lm^AFVMzsWj z(TiLvBg}e)&$9j$EnIep8i!)qi4mnLYi_nq4OLgizpdk3K)sn>v;s)o7Mqypvw{IC zh07KR26z^3Fa!FSoeB)ZJNeaxX(gS26v&Ke%j@ds9;-;?Us$v zJ_y19p_Rvs8K$UczD(l6D%7CknEEkq{`^@KUfLtFeVWHd5loph$M_ZrM}=7Sm7aPLpu>Cw^Ix`gxwow4;^^+w=6qcrx5i&C zoQ9-Z*7*2T^jK9SE2GGqNi}=I)CNgnBw2C?1;_)s9@zi;{+2w5o zDfhfZi{``?@lnOt*6{eE82TRfabfa&6w2G2U$3jlP0egcWX7 z3)1I=70m=<&@6NFcXlg}DjoaCRP$r2yV*%B?x?71CZ%bIUzy8hnT56LzbJ6UkiN|)FwP)*TaEP8Zp9BL%vlC z)vw>XZ0)~l!d#uPNKvV&1CHv6(T#HQ@)o!+h_*KQ4RaLdla@nB$5o#+;>pKd0-GU? zk0{(IW}Hyb_kO73fCnm5qXehi79Ym2gZ*yFQ#4537Yy>EF;CC4s%m*{~E;mlMAAK?ICtRhQa z{vrI&91KJc@>5I*dpAP+S7#(B-RTlvUOw98N8s2s!fy`-hWmvG1T4~Fa-6zNenZWuqyW8lFklWyIofA_ochoQ&;p`pIT zo8K>3`DT19sRn9>nP7a9sf{!+2%$Tm&@skemm;8iI1?EyuU%hI&_M0XH zQ42K3!ysxljR;bh^yXLT!oXgvd;%Uw1=y;H<%RQO3Q0;@J)sC7oL_F=o?3WXY5qei zVgB}KDOqT0p)Rz4|RgJI+byF7RNAh_#cKWkH5dc>sdGiqEWl;s4Yv=>d$&_#*t$(+CCBtDb32n5y z{QU47=ADy3;cuk#wStJ9zcEQ9DXazD`@$TSxp65VVLi3O3pdhE<86O;n(h^T*eQlK zB=h3Vp6zDRb=R|vNke|B4e0hwAP>l$chDpEt&v*XPS<>w3#MZcnR6!(P%ygwb$M#G z@xIBF8fYb1a?I!JDJ>7q`1y0z=7|BPPAN{V4SWKxDq3L9|xPQtoqd(oe z>sn$)OYu;vzImR%P(7o!wsCX|n@J3uz#)w-DdNw@jnz}@(_%)eTEw4H_Ff*-$9b4Z zUZi2UnrTUP|J#K(_jyjb?xi!tB-s{0<;dkHQVUX#G&WfkWp=E*w|f4)F0;~}r^SsJ zM2WhQ61Bej9plm;Rw?GDd1+-GN7~&Rs2yxxozt7lFh%?MLf?t zDkb&5E9D*EqICW(i%Z7;ufq5L^Gf>v=>t=%7B6~#BDLw?Z$H*J0CZ-ql5Jw5+L-?e z`r0W_yK^7ky{a3b>~(36DB@G279TpK7m=JZCCGKIcWbd<5-`_r$4;Jy_@de==PumX z{r}Gm{lCT<{(C}dmk;V{$kX9Kwm4_c#F7cu+q&hsTxg_Cysc?OvewqDn_h0Q)h*TO z@muz!*0l8WXV6;g^+nE6c0YZeI{X=0QJ1#=@4EGG-$3bnT3WgYr4Dr;7z0TSaTwG* zqNpa><(GPTOLMIlOfz@>{F@hlp)@7jsK`D%UOKh4Bgp|pp(;TJ&Jv{pzGHT(USkBs z6qcfF47sEC9434QwN;jumIe~OPouWMF_Eaj$G~|ep)z=8qX9UEH!;BG5xw#YslS_D z>mFIgzexG}sbzWD!!4?EPimqT2HG#AkOVaa42Y*(rsBO(G}5arM*eMYglW+$6t3d% zk|s@RJQ>?U6^=LC*6To&TR*1$jTpa0MbxllcS2j~Gf4aK*RQH1(}HKF7m-hs0iw=~ z!K&5H8#`WBmI_-u_s=Ee2N%+_S~x=!T~}a3gVR48cSEVzmh#uh)z?j{_S^t+ zXrXFC!})HT)8p#2d%;F}L;MXxeYdQrd|X$>-?uy;sFzgLZE;R@>R+2CS9`X+@H3NE-Q2WAzms_- zkIpSmt@C^C(|>j&p}FSOq^Z4YB?+kXSF+2+>?e^(rlw8<)FBPD+P+;B3`hjxUw!8A zA~;eaQ^-;lkZqxxCCep!qv1}+T_|lt~KI{jMA>&?n4Pl{JR|D=-Ybra`Pw zTy$4kEM&+p4z<>$jpV@f8u`q9?rJ`H0RD<9VQdlr%UxZ};YYg%ed3crq1h`V?<$ z;?^jmf#1m)WlkQ3iH5sq%im_wJP%khx$An{g)pwSWYdIkiP1~Baq7cHxH6dw-u#p3zwX~mK`5H)KYTo z)h{$XcuUZqIp!~m_j9nrt|Of!fz2j0Ckl{sz%GNk{YUu9Z%?|Dh8ZUi0g%*WyJ{Ih zOM#&x5LQ845t*|XF%pN}3aDHHrEJ|0NP%@^OD7QTmO}y&ZpxxYhms{sJZT>EexDc@ z*FEszFv;({)_j}P_m@67Ena&LCFLSt5O$qvOLOkuAB@8lc){-7zYY$!yS)4%ke8^2iF$B(CT=S^LIs_k9!kTMZY_TI z)+`fV(Breyg=r4qpTg@~zOSncU8l9!)RTo%D)MMVK4M`pM1nCgkz6$^im|pNs_<-J zB8qw1_9-ElA^Z9H2?r#=OU5{Zv5Bf-V(^xynwKtIKrri1Wz!|#+rx3`M{=x&sHydg zD|$#*mqI_}`xJ^N%9009p7dZwD;;8zu(^z`ri#g~`nH2njQ%8`w2SGG0L-Aa%Z4~4 z{J1$0(2+eYD9FI1kaSo)B8pYaA*S%M#8k)uK8AVNZ+7AK3K&bspAIRLHl9yR1S1## z;>1Q1Ff{+c?eO05l$Jz5M~TFYFF+16YLkZ##fVV>RTDU7FIm!yRmfbn&NTq^iP+vD zo}lRD+*}G{WzgTR9U16GpB+FH=Mm~*sAOU;7}Ws1$)0(8rm;IIdg;64^FrxC>-7q= z<|xm(Au)y>T^>`{Yl`BsvEqF7>LH0U+#trmzXU(abf~(NWgSXLQzri7so(q81(@k6DXmLs;cU3K(Ku= z`yEgX{ij(M1M_(KgkW6e=5;EeL}|O2x$IyQk_Yyo3l$|TKBoYpxZMYwJ}ri?FpFkt<`0F$7`Ph})o z@Rg(U^u}*7QT+P+p_zJ6=HSakHEQxlVSj!N>%OOxch(RY^%)#Kg0&!=$5X0qHu8Y?R90kY&MasLT8}r-t#X40mg`U+Dt>9dhy)(^DIfu zTmE3k3^^va>aA?|m)vj3+O>jb6|fzdhcgRQ0~k*jgcC+V@o3@y-AibblTP48&#PG)SCmMmDi$v%3$=U3=7~#z zA*rm{*XtSU2CAii^qy(!L|Q?oLFZ+)#JtUl%Ugxr$e@;=*}tB6Xl1gGb&`dv3H$_p zt&0ROBbeq_2CAt$d@;e2ash6}Tq@A?Uo`K0z;Y3-aVfw;K50{5fe7&xL;`Ws9P+~c zfB0~;QZC<}(&jpY_?VROKeS{8V*HI}7sB&NsW=45oO4QuRJv7Gf}hKZKXg zU$`)yt$(@J0C!o=|k z>k@+yx?bTgcw)Q}&|aDriFx-7l1NAO0;#Ny4uvI4#ktWT9#-$_TwLd}mAeB1?yCl`*4cp(&bEB)L8Vr|Qu^%bznStJ3ypeb-yP3(l;XNW8*^5~{Abnc0j- zdzpv961c0xpwlS?S~2qhkSNWRp-h*1SVOTeRvEsoZ9q_3)9>G>aCey)+?7v&hjFNQ zGmT&W7Jw}L#tpf|Vu9*I>jlU{Wg;Rexn1j&c?Qoy zyU5_622}>C^5dW;eIN|8=EW!);CTTPgDns;cI=4X9-%fqWg$wlnmz5B^6jU8H{9_0r>}i5YlW~ zoOuLk8lXgCG31>Tl1)FPmg|Oga@etJVQLAo95G3SG>>8rMF2Y*fqo_AG6_01F@+CU zErNoEtA2k@UhLlCc#El#Yyb`d5iI^~T+j&YdqZic`fQjh;W>0!}OSq2j3V_|>FB-1Afj)g(+$mke2$KQLnpGiZ@>87UR$6x3kMTY32p6A?)u%A(;t$9a@R>P z=5KrL>D+Y?uSFXF@s|)^Orr8r95KQJ%?J!T`u4;hC^CXZH-3Bov5LWRJj(NNalV}4 zs5m~>)~=_Wc({ew3;HHH`J+d7A(u_v{Y^wk-Yl{#ztg9s_QHCfK7G$|=W%Lk0@D`l z2ZgexF-g80dLRiOCB;s?LY4tuE=cYM252w3j9&|dB4pFrwV|C(F59!4qpU288s`7> zcIN+BrtKg1OwUs@nMyG#+DMzU)1tBzEmMlg5|!oFDvA&(v>B9W)Du}MOdFw+Qrx7- zl*&k$vV~H~(48bg&--k?f5W$5>J_j1y07ax&*S_YpW}0U4oUAy@D@@Tase?>gh5Sz z{1NoN*gDx#@Z+G$f9B{MqjnBdA>RE`^LCS7>rh#HPmiD8=iC9|M~{+a@}%B>t`BP! zmIjjiL#ioc-3PwO{6)uZaV=*il|zvOpy5xpTn55L(FWkBI2VbPJ`F(?6>~N38Lsm^ z(|JGL=I@Lt5>Vjw?2fT`Ef|4j(w+f=61tuwras%O{iM8nG1XS)uEuyz_L2HMbvHHM z`I} zd!yh?wU#B?Sr7L}OG`6T`p>cZdC<54{?CgutPHrcCpY$9mAsFZW+sdqGkUt)Cnlh>XYVR6_^G^)q+Xnb=G zg4B)goSvaJ$+J%>dCLM<{pnaj{0hydx1yT7Ubfj(>+`*y|oRY3-@teWTYNoH4% z$*g)zQ3Zc)b?RrJq(h&_gDplW2p{Pd|2z?4cYPlz_UAJGMSdM-1gy=&?URe$Wg3x5Prvq2AVG`_C)$(gy)JWg_0N|H$Q+4<8F`UWh(-!>-> z{iea^i;9B=iD3xHe{yQ3n7gBzBYhK7cuz@OU#P7!tMnX>o}so|co!~e!Imu-BLiUhM9M(9*2K@Rr(d}-tMg590(BLE zd%)zJbyTHd27>3Q5NI8P3`Q8F!G>~1aKEQ9&Q}DQ94^M;AZ~MC^`K2 z3noRBIx+O-+t_kz>-#`VY87`|+TB?kqB6$O#Jr1fbLTR0JfH9mK~#^+u&5^xayYf; zfNwa@nJOZ>r-fn$+bFh`s4)^=&>NDuk!i6I#m?h#;>Rb-DQrcOo!j^I#U?$A>Oz;M1A5(azhp5rzbSKCjP-a|aBL@aOT_u=df+FyGel2R?fNC%xgMqr(u12UEIT^BZl7c z$f1=0S>Z^0_)0>~j<1CfK5(nV2)h;$}ZPIUv z4a;;~6{~kta1=Z2%rdJRom_8n2T=ha)iN6u4o7`JlB2!-UOGtW*GQ9KwO907Oif2( z28&D{E;%trbhVQTdV5Dg5u(^qQ`-j}!vZ%xvl&25YD~Vzd`;fCx~m7D20z8-J`z7( zn4SVB(VxL-yhIf}cK-ZWloNCnZER7ZB#~S{y0g;l@_<~K%R8dTc%Fx0s_XN!&GU*9qELp zr4B-3wJt1Q0zps=s^U8f^u2yMDk>9jA>WQY56LIO0bCg$#o}QzDVOyxH5uMNkbIhA z%T#S`p&UW(m?yDnm$JBo#DtxTfSdRyyosMZo1pc`DJDu~;_qdFjT_&+eXBNWR>bP2 z#wQze40aJ)=6(Egp~T`b>BDEHH?@X#R=oY=+Cb%UNz;z&BoDALVjR!vnVWA^1kBO# zGqCgzwbPt=YK*$yv*+n2HiULp|Fv(a&Wvou{|q1by>gC{R`xJiL$mkDOPb+zg>!5# zH>@wzDM*=OHFr$g*)T7cyz*A>4y$*I9;MP5vq}F8cRCaW6T%kppy18PUA@ga>NZId zw;Fo>u-rV=sk&6xwDrQFWu@J5x3-RT_saj0ZSh-tS*ne{tZ4D0(^2O-GaG$^>~@|n zTf8TFRqLEjLocRHx6nxXZp}K)#HhH^FBdj?r1vj}xV~K_Q;W9E((4ybn03$3-X3`=EVw3QlHR(u>c-{z*$*9q zS9)c=jz3|N?4N!lu{|uc7u zBjL9@l0kLlQ<9GCX?h$G9=$B&2-boRq5ryFB{I-gqa_X*mI&~vSac^C%bqX&6lkidZ^g5Hkr*0zF#?A zQ#RM6I>@-ua-eeKSg0517232oPk+x%;_Q?@9+cG)iDid9YAuWAb#uIRO6TVp8* z{K)giKI-sCI$o~kU|KPnmpyP9pN4_fe$ur?>9UjzZ}ei1%wES`C`Xos4#u}?S4Q<7 zA@x5pYLaq^LR;6sw&PYn8$w=w&k)f+v<*UC^gaL-08livk0F;VO|o*F87+EtD3l2< z3tCF}fIv41+z4qQG#8L5lqaziGK(Q&A_QTwmnoC z!J7o;D>YxqWVx*5w0~yR3ih&1^3FT%)=&RBNB|z;_t^oU*2{p{jfVmyr)fx39&Hqr zhXc8ShaEh{HV@mjsZ$N8#}Ftb`lPnKK@cRe5GW-4OJFJCy5X_hEb~#8lEz3BlVDy5 z$KWEOHF0aGii-NISys-I2}t<_8uU5n=XtOEwtxzvmUHab=m%#z->Isq{?iwa#k#ml z%One}g_E?jvbGuGiJ(!wz((sFy%31-R&q8YqNEmgcIcB5><6eEtRz5q zgq@$LT7a%auMm}ppdnA6_Hz-u7Zq-Tt6k5B2tlAj#ouA4UrxI11s* zP6>_^gfyjNsaFry20~J9WxFeYXO4#X|0Y~XOh5TZDKQ+9D&1SSu7ghfBB6=@^UtH7 zyBNjC7%W={bi^Y0I;P=MD39S-$%@4B`;Tkz00f#;;|oPa9*t9K zPLD1xO8{BQHfad2#7v-7*47g~DT~q_qNv;TviGcQ5dDy0!+uXlH~_rhU`0@eavw(; zh3|vzo_&!#-xSH(oE5RU0dU-alm;k!K%^iPc%o`2%9gU2&DF$WhUgowB=PoM9a9zw zCBb5#7Kj|}P{|LTy2C|o&s$u%r%n;LaCnk5Gc)u4g9mq7+Hpz{|M(`05Ta%HnSs0g zgo+kHGCWs)@rTCHfX3#1ixLL%MnMWAyWcDxSLvc~n<^ZwSv@t_2J_s7zbziN?kpxj z*L}^A?(Wzk2}_Xkv(fN1IG`AJO>@lVtr}Z`rWG#=o|fCM_`kvOgKw3@`VUf_)$C!l z)OmyTU_FIvZ-xxh{m;U{y-x}rUYjvVDnBDyN7;1Vd4(Ap?qE z`D)XRF|y3|r9bv1ncSm21}y@_6EKE7-bK zKHbJ7@85HC>NaiKG;P+b!2En;oX{?%rXKdL`uXgBl*9y1W>3fea?#+#24`pIplZ%`B$XYu4N)g`sfg*4^oOrN~|%l8AJh2#-S&KV7JII&g#a zQH>vxl9D7Xi1>5nkL23NW8uH{eEHNbb_M=qnEqU=aJi7EDS7sj@xapO{H}U`z1S6_ zBMRgA0%l4tcg&%fb0P6is$Gu6(Gl6GJU*@jxLYZXyLYQbon!bse$PJA(7*q7!Gnom zemH9)HyfFVr{ zsU{5E?ZubWDwk7OdbYi#CQz%e+V^eHn?Gl3k6Hh5%ITs&hUL_9CLP@{^rCrR=9qkz z)kA6y89qFqrlo+ID2Kt`;$pMqEATp5a%Syh>QfYARM{~~s_5a>@7=q1^yjk_=dRqi zF>b`xsIiDr7<|vULE2((t=u=@yIWP0$&>J@!(+G)j?`z&HJNKV`q=%Z%!9RSViKpaM2cT|%Tpe1e~N9E!lbxcDTn)=)L%7TX{x?{n2!|W&dV7YK_F*|gTue% z7%F1y+5PgRhMs><2=l%~M>Pvd`7jZWNjte$wO+RFGEcHto4KEWcy;CF z9eiWEtGiQ2W9gvby}24oHS*-i+^(*9Cn6%uCPy$5z}zAqmgGgNZz}$at?}N=O0>1K zv~FM3;^Fi9)=v*6%7%o6C76zX0Rd0dJ-4ap(wp0Kpe;&!c2KJjZA9c>FPti7@{w*R zcb9~}>>hkHsHc3$QO_kew-4}a-!G?7^<8sMFR#S`t@@A0_`hELC10t1r|#^+k=^v?W?03S=T+Ukn%(PSrg((~m0WpYTj!b+QQcZr zp}NfUn^7ZR^`^hnmZaA^~ulO-*Y#~x#xi1 z>zZK+H|8~O56i5vubutJyUyt9@7qTe&t2bSwYq%ni|+dhWnZ5AmA757lMNqOeENmG tp^=f%soIr;)Ya8L{MbkS|L3oV`mUYE<$VXaU6$j|QsWgyDGTiP{|^_iZE^qr literal 0 HcmV?d00001 diff --git a/ea/ForecastingNewsletter/April2020.md b/ea/ForecastingNewsletter/April2020.md index 9285cdb..be0bacd 100644 --- a/ea/ForecastingNewsletter/April2020.md +++ b/ea/ForecastingNewsletter/April2020.md @@ -1,13 +1,17 @@ -Whatever happened to forecasting? April 2020 -============================================ +A forecasting digest with a focus on experimental forecasting. +- You can sign up [here](https://mailchi.mp/18fccca46f83/forecastingnewsletter). +- You can also see this post on LessWrong [here](https://www.lesswrong.com/posts/e4C7hTmbmPfLjJzXT/forecasting-newsletter-april-2020-1) +- And the post is archived [here](https://nunosempere.github.io/ea/ForecastingNewsletter/) -A forecasting digest with a focus on experimental forecasting. -- You can sign up [here](https://mailchi.mp/18fccca46f83/forecastingnewsletter). -- You can also see this post on the [EA forum](https://forum.effectivealtruism.org/posts/9YJKugJ68qTFzMNCM/forecasting-newsletter-april-2020) or on LessWrong [here](https://www.lesswrong.com/posts/e4C7hTmbmPfLjJzXT/forecasting-newsletter-april-2020-1). +The newsletter itself is experimental, but there will be at least five more iterations. Feel free to use this post as a forecasting open thread. -The newsletter itself is experimental, but there will be at least five more iterations. +Why is this relevant to EAs? +- Some items are immediately relevant (e.g., forecasts of famine). +- Others are projects whose success I'm cheering for, and which I think have the potential to do great amounts of good (e.g., Replication Markets). +- The remaining are relevant to the extent that cross-polination of ideas is valuable. +- Forecasting may become/is becoming a powerful tool for world-optimization, and EAs may want to avail themselves of this tool. -Conflict of interest: With Foretold in general and Jacob Laguerros in particular. This is marked as (c.o.i) throughout the text. +Conflict of interest: Marked as (c.o.i) throughout the text. ## Index - Prediction Markets & Forecasting platforms. diff --git a/ea/ForecastingNewsletter/Header.md b/ea/ForecastingNewsletter/Header.md new file mode 100644 index 0000000..5d98a64 --- /dev/null +++ b/ea/ForecastingNewsletter/Header.md @@ -0,0 +1,19 @@ +An opinionated forecasting digest with a focus on experimental forecasting. +- You can sign up [here](https://mailchi.mp/18fccca46f83/forecastingnewsletter). +- You can also see this post on LessWrong [here]() +- And the post is archived [here](https://nunosempere.github.io/ea/ForecastingNewsletter/) + +The newsletter itself is experimental, but there will be at least four more iterations. + +Feel free to use this post as a forecasting open thread. + +Why is this relevant to Effective Altruism? +- Some items are immediately relevant (e.g., forecasts of famine). +- Others are projects whose success I'm cheering for, and which I think have the potential to do great amounts of good (e.g., Replication Markets). +- The remaining are relevant to the extent that cross-polination of ideas is valuable. +- Forecasting may become a powerful tool for world-optimization, and EAs may want to avail themselves of this tool. + +In short, in the words of a sect of the Sith: "Through knowledge, I gain strength; through strength, I gain power; through power, I gain victory. Through victory, my chains are broken." + +Conflicts of interest: Marked as (c.o.i) throughout the text. +Note to the future: All links are added automatically to the Internet Archive. In case of link rot, go [here](https://archive.org/) diff --git a/ea/ForecastingNewsletter/May2020.md b/ea/ForecastingNewsletter/May2020.md new file mode 100644 index 0000000..e8f987d --- /dev/null +++ b/ea/ForecastingNewsletter/May2020.md @@ -0,0 +1,199 @@ +Whatever happened to forecasting? April 2020 +============================================ + +A forecasting digest with a focus on experimental forecasting. You can sign up [here](https://mailchi.mp/18fccca46f83/forecastingnewsletter). The newsletter itself is experimental, but there will be at least five more iterations. + +## Index +- Prediction Markets & Forecasting platforms. + - Augur. + - PredictIt & Election Betting Odds. + - Replication Markets. + - Coronavirus Information Markets. + - Foretold. (c.o.i). + - Metaculus. + - Good Judgement Open. +- In the News. +- Long Content. + +## Prediction Markets & Forecasting platforms. + +### Augur: [augur.net](https://www.augur.net/) + +Augur is a decentralized prediction market. [Here](https://bravenewcoin.com/insights/augur-price-analysis-token-success-hinges-on-v2-release-in-june) is a fine piece of reporting outlining how it operates and the road ahead. + +### Predict It & Election Betting Odds: [predictIt.org](https://www.predictit.org/) & [electionBettingOdds.com](http://electionbettingodds.com/) + +PredictIt is a prediction platform restricted to US citizens or those who bother using a VPN. This month, they have a badass map about the election college result in the USA. States are colored according to the market prices: + +![](https://nunosempere.github.io/ea/Forecasting/electoral_college_predictit.png) + +Some of the predictions I found most interesting follow. The market probabilities can be found below; the engaged reader might want to annotate their probabilities and then compare. +- [Will Benjamin Netanyahu be prime minister of Israel on Dec. 31, 2020?](https://www.predictit.org/markets/detail/6238/Will-Benjamin-Netanyahu-be-prime-minister-of-Israel-on-Dec-31,-2020) +- [Will Trump meet with Kim Jong-Un in 2020?](https://www.predictit.org/markets/detail/6265/Will-Trump-meet-with-Kim-Jong-Un-in-2020) +- [Will Nicolás Maduro be president of Venezuela on Dec. 31, 2020?](https://www.predictit.org/markets/detail/6237/Will-Nicol%C3%A1s-Maduro-be-president-of-Venezuela-on-Dec-31,-2020) +- [Will Kim Jong-Un be Supreme Leader of North Korea on Dec. 31?](https://www.predictit.org/markets/detail/6674/Will-Kim-Jong-Un-be-Supreme-Leader-of-North-Korea-on-Dec-31) +- [Will a federal charge against Barack Obama be confirmed before November 3?](https://www.predictit.org/markets/detail/6702/Will-a-federal-charge-against-Barack-Obama-be-confirmed-before-November-3) + +Some of the interesting and wrong ones are: +- [Will Trump switch parties by Election Day 2020?](https://www.predictit.org/markets/detail/3731/Will-Trump-switch-parties-by-Election-Day-2020) +- [Will Michelle Obama run for president in 2020?](https://www.predictit.org/markets/detail/4632/Will-Michelle-Obama-run-for-president-in-2020) +- [Will Hillary Clinton run for president in 2020?](https://www.predictit.org/markets/detail/4614/Will-Hillary-Clinton-run-for-president-in-2020) + +Answers are: 80%, 15%, 69%, 79%, 8%, 2%, 7%, 11%. + +Further, the following two markets are plain inconsistent: +- [Will the 2020 Democratic nominee for president be a woman?](https://www.predictit.org/markets/detail/2902/Will-the-2020-Democratic-nominee-for-president-be-a-woman): 11% +- [Who will win the 2020 Democratic presidential nomination?](https://www.predictit.org/markets/detail/3633/Who-will-win-the-2020-Democratic-presidential-nomination). Biden, Cuomo and Sanders sum up to 95%. + +[Election Betting Odds](https://electionbettingodds.com/) aggregates PredictIt with other such services for the US presidential elections. The creators of the webpage used its visibility to promote [ftx.com](https://ftx.com/), another platform in the area. They also have an election map. + +### Replication Markets: [replicationmarkets.com](https://www.replicationmarkets.com) +Replication Markets is a project where volunteer forecasters try to predict whether a given study's results will be replicated with high power. Rewards are monetary, but only given out to the top N forecasters, and markets suffer from sometimes being dull. + +The first week of each round is a survey round, which has some aspects of a Keynesian beauty contest, because it's the results of the second round, not the ground truth, what is being forecasted. This second round then tries to predict what would happen if the studies were in fact subject to a replication, which a select number of studies then undergo. + +There is a part of me which dislikes this setup: here was I, during the first round, forecasting to the best of my ability, when I realize that in some cases, I'm going to improve the aggregate and be punished for this, particularly when I have information which I expect other market participants to not have. + +At first I thought that, cunningly, the results of the first round are used as priors for the second round, but a programming mistake by the organizers revealed that they use a simple algorithm: claims with p < .001 start with a prior of 80%, p < .01 starts at 40%, and p < .05 starts at 30%. + +### Coronavirus Information Markets: [coronainformationmarkets.com](https://coronainformationmarkets.com/) +For those who want to put their money where their mouth is, a prediction market for coronavirus related information popped out. + +Making forecasts is tricky, so would-be-bettors might be better off pooling their forecasts. As of the middle of this month, the total trading volume sits at a $20k (from 8k last month), and some questions have been resolved already. + +### Foretold: [foretold.io](https://www.foretold.io/) & EpidemicForecasting (c.o.i) + +Foretold has continued their partnership with Epidemic Forecasting, gathering a team of superforecasters to advise governments around the world which wouldn't otherwise have the capacity. They further shipped a report to a vaccine company analyzing the suitability of different locations for human trials, aggregating more than 1000 individual forecasts. + +### Metaculus: [metaculus.com](https://www.metaculus.com/) + +Metaculus is a forecasting platform with an active community and lots of interesting questions. In their May pandemic newsletter, they emphasized having "all the benefits of a betting market but without the actual betting", which I found pretty funny. + +Yet consider that if monetary prediction markets were more convenient to use, and less dragged down by regulatory hurdles in the US, they could have been scaled up much more quickly during the pandemic. + +Instead, the job fell to Metaculus; this month they've organized a flurry of activities, most notably: +- [The Salk Tournament](https://pandemic.metaculus.com/questions/4093/the-salk-tournament-for-coronavirus-sars-cov-2-vaccine-rd/) on vaccine development +- [The El Paso Series](https://pandemic.metaculus.com/questions/4161/el-paso-series-supporting-covid-19-response-planning-in-a-mid-sized-city/) on collaboratively predicting peaks. +- [The Lightning Round Tournament](https://pandemic.metaculus.com/questions/4166/the-lightning-round-tournament-comparing-metaculus-forecasters-to-infectious-disease-experts/), in which metaculus forecasters go head to head against expert epidemiologists. +- They also present a [Covid dashboard](https://pandemic.metaculus.com/COVID-19/). + +On the negative side, they haven't fixed the way users input their distribution, restricting it to stacking up to 5 gaussians on top of each other, which limits expressiveness. + +### /(Good Judgement?[^]*)|(Superforecast(ing|er))/gi + +The title of this section is a [regular expression](https://en.wikipedia.org/wiki/Regular_expression), so as to be maximally unambiguous. + +Good Judgement Inc. is the organization which grew out of Tetlock's research on forecasting, and out of the Good Judgement Project, which won the [IARPA ACE forecasting competition](https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation_(ACE)_Program), and resulted in the research covered in the *Superforecasting* book. + +Good Judgement Inc. also organizes the Good Judgement Open [gjopen.com](https://www.gjopen.com/), a forecasting platform open to all, with a focus on serious geopolitical questions. They structure their questions in challenges. +- [Before 1 January 2021, will the People's Liberation Army (PLA) and/or People’s Armed Police (PAP) be mobilized in Hong Kong?](https://www.gjopen.com/questions/1499-before-1-january-2021-will-the-people-s-liberation-army-pla-and-or-people-s-armed-police-pap-be-mobilized-in-hong-kong) +- [Will the winner of the popular vote in the 2020 United States presidential election also win the electoral college?](https://www.gjopen.com/questions/1495-will-the-winner-of-the-popular-vote-in-the-2020-united-states-presidential-election-also-win-the-electoral-college)- This one is interesting, because it has infrequently gone the other way historically, but 2/5 of the last USA elections were split. +- [Will Benjamin Netanyahu cease to be the prime minister of Israel before 1 January 2021?](https://www.gjopen.com/questions/1498-will-benjamin-netanyahu-cease-to-be-the-prime-minister-of-israel-before-1-january-2021). Just when I thought he was out, he pulls himself back in. +- [Before 28 July 2020, will Saudi Arabia announce the cancellation or suspension of the Hajj pilgrimage, scheduled for 28 July 2020 to 2 August 2020?] (https://www.gjopen.com/questions/1621-before-28-july-2020-will-saudi-arabia-announce-the-cancellation-or-suspension-of-the-hajj-pilgrimage-scheduled-for-28-july-2020-to-2-august-2020) +- [Will formal negotiations between Russia and the United States on an extension, modification, or replacement for the New START treaty begin before 1 October 2020?](https://www.gjopen.com/questions/1551-will-formal-negotiations-between-russia-and-the-united-states-on-an-extension-modification-or-replacement-for-the-new-start-treaty-begin-before-1-october-2020)s + +On the Good Judgement Inc. side, [here](https://goodjudgment.com/covidrecovery/) is a dashboard presenting forecasts related to covid. The ones I found most worthy are: +- [When will the FDA approve a drug or biological product for the treatment of COVID-19?](https://goodjudgment.io/covid-recovery/#1384) +- [Will the US economy bounce back by Q2 2021?](https://goodjudgment.io/covid-recovery/#1373) +- [What will be the U.S. civilian unemployment rate (U3) for June 2021?](https://goodjudgment.io/covid-recovery/#1374) +- [When will enough doses of FDA-approved COVID-19 vaccine(s) to inoculate 25 million people be distributed in the United States?](https://goodjudgment.io/covid-recovery/#1363) + +Otherwise, for a recent interview with Tetlock, see [this podcast](https://medium.com/conversations-with-tyler/philip-tetlock-tyler-cowen-forecasting-sociology-30401464b6d9), by Tyler Cowen. + +## CSET: Foretell + +The Center for Security and Emerging Technology is looking for forecasters to predict the future to better inform policy decisions. For a more elaborate explanation, and to sign-up when applications open, see [their webpage](https://www.cset-foretell.com/). CSET was previously funded by the [Open Philantropy Project](https://www.openphilanthropy.org/giving/grants/georgetown-university-center-security-and-emerging-technology), and seems to have a legibly impressive leadership lineup. + +## In the News +- [In Forecasting Hurricane Dorian, Models Fell Short](https://www.scpr.org/news/2020/04/30/92263/in-forecasting-hurricane-dorian-models-fell-short/) (and see [here](https://www.nhc.noaa.gov/data/tcr/AL052019_Dorian.pdf) for the National Hurricane Center report). "Hurricane forecasters and the models they depend on failed to anticipate the strength and impact of last year's deadliest storm." +- [The Post ranks the top 10 faces in New York sports today](https://nypost.com/2020/05/02/the-post-ranks-the-top-10-faces-in-new-york-sports-today/), accompanied by [Pitfall to forecasting top 10 faces of New York sports right now](https://nypost.com/2020/05/03/pitfall-to-forecasting-top-10-faces-of-new-york-sports-right-now/). Comparison with the historical situation: Check. Considering alternative hypothesis: Check. Communicating uncertainty to the reader in an effective manner: Check. Putting your predictions out to be judged: Check. +- Kings College produces a new [forecasting tool for central banks](https://www.kcl.ac.uk/news/new-covid-19-relating-forecasting-tool-central-banks-2) +- [Nounós Creamery uses demand-forecasting platform to improve production process](https://www.dairyfoods.com/articles/94319-noun%C3%B3s-creamery-uses-demand-forecasting-platform-to-improve-production-process). The piece is shameless advertising, but it's still an example of predictive models used out in the wild in industry. +- [Nowcasting and Forecasting of COVID-19](https://www.mrc-bsu.cam.ac.uk/tackling-covid-19/nowcasting-and-forecasting-of-covid-19/), from the University of Cambridge. Sadly solely for England, which has a great bureaucracy which can presumably track most if not all covid deaths. +- [BMW Cuts Profit Forecast Again, And Warns About Uncertainty](https://www.forbes.com/sites/neilwinton/2020/05/06/bmw-cuts-profit-forecast-again-and-warns-about-uncertainty/#2ac2be64468c), Forbes reports. +- [Central Bankers Adopt Scenario Forecasting for Post-Virus World](https://www.bloomberg.com/news/articles/2020-05-11/central-bankers-adopt-scenario-forecasting-for-post-virus-world). I find it cute that China, seeing as how they're not going to be able to meet their GDP targets, is "considering dropping its traditional numerical GDP target". Otherwise, central banks are coming to terms with the depths of their uncertainty. +- [Locust-tracking application for the UN](https://www.research.noaa.gov/article/ArtMID/587/ArticleID/2620/NOAA-teams-with-the-United-Nations-to-create-locust-tracking-application). (and [here](https://www.washingtonpost.com/weather/2020/05/13/east-africa-locust-forecast-tool/) is a take by the Washington Post), using software originally intended to track the movements of air polution. NOAA also sounds like a really cool organization: "NOAA Research enables better forecasts, earlier warnings for natural disasters, and a greater understanding of the Earth. Our role is to provide unbiased science to better manage the environment, nationally, and globally." +- [United Nations: World Economic Situation and Prospects as of mid-2020](https://www.un.org/development/desa/dpad/publication/world-economic-situation-and-prospects-as-of-mid-2020/). A recent report is out, which predicts a 3.2% contraction of the global economy. Between 34 and 160 million people are expected to fall below the extreme poverty line this year. +- [Kelsey Piper of Vox disses on the IHME model](https://www.vox.com/future-perfect/2020/5/2/21241261/coronavirus-modeling-us-deaths-ihme-pandemic). "Some of the factors that make the IHME model unreliable at predicting the virus may have gotten people to pay attention to it;"or "Other researchers found the true deaths were outside of the 95 percent confidence interval given by the model 70 percent of the time." +- [Fox News](https://www.fox10phoenix.com/news/cdc-says-all-models-forecast-increase-in-covid-19-deaths-in-coming-weeks-exceeding-100k-by-june-1) and [Business Insider](https://www.businessinsider.com/cdc-forecasts-100000-coronavirus-deaths-by-june-1-2020-5?r=KINDLYSTOPTRACKINGUS) report over the CDC forecasting 100k deaths by June the 1st, differently. +- Yahoo has automated finance forecast reporting. It took me a while (three months) to notice that the low quality finance articles that were popping up in my google alerts were machine generated. See [Synovus Financial Corp. Earnings Missed Analyst Estimates: Here's What Analysts Are Forecasting Now](https://finance.yahoo.com/news/synovus-financial-corp-earnings-missed-152645825.html), [Wienerberger AG Earnings Missed Analyst Estimates: Here's What Analysts Are Forecasting Now](https://finance.yahoo.com/news/wienerberger-ag-earnings-missed-analyst-070545629.html), [Park Lawn Corporation Earnings Missed Analyst Estimates: Here's What Analysts Are Forecasting Now](https://news.yahoo.com/park-lawn-corporation-earnings-missed-120314826.html); they have a similar structure, paragraph per paragraph, and seem to have been generated from a template which changes a little bit depending on the data (they seem to have different templates for very positive, positive, neutral and negative change). To be clear, I could program something like this given a good finance api and a spare week/month, and in fact did so a couple of years ago for an automatic poetry generator. *But I didn't notice because I wasn't paying attention*. +- [Sports betting alternatives which are booming during the Corona shutdown](https://thegamehaus.com/sports/sports-betting-alternatives-which-are-booming-during-the-corona-shutdown/2020/05/15/). Suggested alternatives for bettors include e-sports, casinos, politics and reality-tv. +- Some transcient content on 538 about [Biden vs past democratic nomines](https://fivethirtyeight.com/features/how-does-biden-stack-up-to-past-democratic-nominees/), about [Trump vs Biden polls](https://fivethirtyeight.com/features/you-can-pay-attention-to-those-trump-vs-biden-polls-but-be-cautious/) and about [the USA vicepresidential draft](https://fivethirtyeight.com/features/its-time-for-another-2020-vice-presidential-draft/), and an old [review of the impact of VP candidates in USA elections](http://baseballot.blogspot.com/2012/07/politically-veepstakes-isnt-worth.html) which seems to have aged well. 538 also brings us this overview of [models with unrealistic-yet-clearly-stated assumptions](https://projects.fivethirtyeight.com/covid-forecasts/); apparently, deaths (not "confirmed deaths", just "deaths") according to John Hopkins University are flat out *linear* from April 1 to May 1. +- [Why Economic Forecasting Is So Difficult in the Pandemic](https://hbr.org/2020/05/why-economic-forecasting-is-so-difficult-in-the-pandemic). Harvard Review Economists share their difficulties. Problems include "not knowing for sure what is going to happen", the government passing legislation unusually fast, sampling errors and reduced response rates from surveys, and lack of knowledge about epidemiology. +- [Nowcasting the weather in Africa](https://phys.org/news/2020-05-storm-chasers-life-saving.html) to reduce fatalities. +- The [Washington post](https://www.washingtonpost.com/outlook/2020/05/19/lets-check-donald-trumps-chances-getting-reelected/) offers a highly partisan view of Trump's chances of winning the election. The author, having already made a past prediction, and seeing as how other media outlets offer a conflicting perspective, rejects the information he's learnt, and instead can only come up with reasons which confirm his initial position. Problem could be solved with a prediction market. +- [IBM releases new AI forecasting tool](https://www.ibm.com/products/planning-analytics): "IBM Planning Analytics is an AI-infused integrated planning solution that automates planning, forecasting and budgeting. By accelerating processes and obtaining more reliable results, Planning Analytics powers more intelligent workflows that drive greater accuracy and efficiency. Now you can quickly and easily drive faster, more accurate plans for financial operations, sales, supply chain and beyond. And with the new release of the Planning Analytics On Demand version, small to medium businesses can transcend the limits of spreadsheet planning and drive better business decisions at a lower price point." See [here](https://www.channelasia.tech/article/679887/ibm-adds-ai-fuelled-forecasting-planning-analytics-platform/) or [here](https://www.cio.com/article/3544611/ibm-adds-ai-fueled-forecasting-to-planning-analytics-platform.html) for a news take. + [Auditor urges more oversight, better forecasting at the United State's Department of Transport](https://www.wral.com/coronavirus/auditor-urges-more-oversight-better-forecasting-at-dot/19106691/): "Instead of basing its spending plan on project-specific cost estimates, Wood said, the agency uses prior-year spending. That forecasting method doesn't account for cost increases or for years when there are more projects in the works." The budget of the organization is $5.9 billion. Problem could be solved with a prediction market. +- [California politics pretends to be about recession forecasts](https://calmatters.org/economy/2020/05/newsom-economic-forecast-criticism-california-model-recession-budget/). Problem could be solved with a prediction market. See also: [Simulacra levels](https://www.lesswrong.com/posts/fEX7G2N7CtmZQ3eB5/simulacra-and-subjectivity?commentId=FgajiMrSpY9MxTS8b); the article is on simulacrum level 3. Key quote, about a given forecasting model: "It’s just preposterously negative... How can you say that out loud without giggling?" + +## Grab bag +- [SlateStarCodex](https://slatestarcodex.com/2020/04/29/predictions-for-2020/) brings us a hundred more predictions for 2020. Some analysis by Zvi Mowshowitz [here](https://www.lesswrong.com/posts/gSdZjyFSky3d34ySh/slatestarcodex-2020-predictions-buy-sell-hold) and by [Bucky](https://www.lesswrong.com/posts/orSNNCm77LiSEBovx/2020-predictions). +- [FLI Podcast: On Superforecasting with Robert de Neufville](https://futureoflife.org/2020/04/30/on-superforecasting-with-robert-de-neufville/). Leaning towards introductory, broad and superficial; I would have liked to see a more intense drilling on some of the points. It still gives pointers to interesting stuff, though, chiefly [The NonProphets Podcast](https://nonprophetspod.wordpress.com/), which looks like it has some more in-depth stuff. Some quotes: +> So it’s not clear to me that our forecasts are necessarily affecting policy. Although it’s the kind of thing that gets written up in the news and who knows how much that affects people’s opinions, or they talk about it at Davos and maybe those people go back and they change what they’re doing. +> I wish it were used better. If I were the advisor to a president, I would say you should create a predictive intelligence unit using superforecasters. Maybe give them access to some classified information, but even using open source information, have them predict probabilities of certain kinds of things and then develop a system for using that in your decision making. But I think we’re a fair ways away from that. I don’t know any interest in that in the current administration. +> Now one thing I think is interesting is that often people, they’re not interested in my saying, “There’s a 78% chance of something happening.” What they want to know is, how did I get there? What is my arguments? That’s not unreasonable. I really like thinking in terms of probabilities, but I think it often helps people understand what the mechanism is because it tells them something about the world that might help them make a decision. So I think one thing that maybe can be done is not to treat it as a black box probability, but to have some kind of algorithmic transparency about our thinking because that actually helps people, might be more useful in terms of making decisions than just a number. +- [Forecasting s-curves is hard](https://constancecrozier.com/2020/04/16/forecasting-s-curves-is-hard/): Some sweet visualizations of what it says on the title. +- [Fashion Trend Forecasting](https://arxiv.org/pdf/2005.03297.pdf) using Instagram and baking preexisting knowledge into NNs. +- [Space Weather Challenge and Forecasting Implications of Rossby Waves](https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2018SW002109). Recent advances may help predict solar flares better. I don't know how bad the worst solar flare could be, and how much a two year warning could buy us, but I view developments like this very positively. +- [The advantages and limitations of forecasting](https://rwer.wordpress.com/2020/05/12/the-advantages-and-limitations-of-forecasting/). A short and sweet blog post, with a couple of forecasting anecdotes and zingers. +- The [University of Washington Medicine](https://patch.com/washington/seattle/uw-medicine-forecasting-losses-500-million-summers-end) might be pretending they need more money to try to bait donors. Of course, America being America, they might actually not have enough money. During a pandemic. "UW Medicine has been at the forefront of the national response to COVID-19 in treating critically ill patients". +- [Forecasting drug utilization and expenditure: ten years of experience in Stockholm](https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-05170-0). A normally pretty good forecasting model had the bad luck of not foreseeing a Black Swan, and sending a study to a journal just before a pandemic, so that it's being published now. They write: "According to the forecasts, the total pharmaceutical expenditure was estimated to increase between 2 and 8% annually. Our analyses showed that the accuracy of these forecasts varied over the years with a mean absolute error of 1.9 percentage points." They further conclude: "Based on the analyses of all forecasting reports produced since the model was established in Stockholm in the late 2000s, we demonstrated that it is feasible to forecast pharmaceutical expenditure with a reasonable accuracy." Presumably, this has increased further because of covid, sending the mean absolute error through the roof. +- In this time of need, where global cooperation might prove to be immensely valuable, Italy has lessons to share about how to forecast the coronavirus. The article [Forecasting in the Time Of The Coronavirus](https://www.bancaditalia.it/media/notizie/2020/en_Previsioni_al_tempo_del_coronavirus_Locarno_Zizza.pdf), by the Central Bank of Italy, is only available in Italian. Mysteriously, the press release, however, is in [English](https://www.bancaditalia.it/media/notizia/forecasting-in-the-time-of-coronavirus/). +- [An analogy-based method for strong convection forecasts in China using GFS forecast data](https://www.tandfonline.com/doi/full/10.1080/16742834.2020.1717329). "Times in the past when the forecast parameters are most similar to those forecast at the current time are identified by searching a large historical numerical dataset", and this is used to better predict one particular class of meteorological phenomena. See [here](https://www.eurekalert.org/pub_releases/2020-05/ioap-ata051520.php) for a press release. +- Some interesting discussion about forecasting over at Twitter, in [David Manheim](https://twitter.com/davidmanheim)'s, [Philip Tetlock](https://twitter.com/PTetlock)'s accounts, some of which have been incorporated into this newsletter. [This twitter thread](https://twitter.com/lukeprog/status/1262492767869009920) contains some discussion about how Good Judgement Open, Metaculus and expert forecasters fare against each other. In particular, note the caveats by @LinchZhang: "For Survey 10, Metaculus said that question resolution was on 4pm ET Sunday, a lot of predictors (correctly) gauged that the data update on Sunday will be delayed and answered the letter rather than the spirit of the question (Metaculus ended up resolving it ambiguous). [This thread](https://twitter.com/mlipsitch/status/1257857079756365824) by Marc Lipsitch has become popular, and I personally also enjoyed [these](https://twitter.com/LinchZhang/status/1262127601176334336) [two](https://twitter.com/LinchZhang/status/1261427045977874432) twitter threads by Linchuan Zhang, on forecasting mistakes. +- The Cato Institute releases [12 New Immigration Ideas for the 21st Century](https://www.cato.org/publications/white-paper/12-new-immigration-ideas-21st-century), including two from Robin Hanson: Choosing Immigrants through Prediction Markets & Transferable Citizenship. +- [Forecasting the Weather in 1946](https://www.smh.com.au/environment/weather/from-the-archives-1946-forecasting-the-world-s-weather-20200515-p54tfd.html) +- Some films are so bad it's funny. [This article fills the same niche](https://www.moneyweb.co.za/investing/yes-it-is-possible-to-predict-the-market/) for forecasting. It has it all: Pythagorean laws of vibration, epicycles, an old and legendary master with mystical abilities, 90 year predictions which come true. Further, from the [Wikipedia entry](https://en.wikipedia.org/wiki/William_Delbert_Gann#Controversy): "He told me that his famous father could not support his family by trading but earned his living by writing and selling instructional courses." +- [A General Approach for Predicting the Behavior of the Supreme Court of the United States](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2463244). What seems to be a pretty simple algorithm (a random forest!) seems to do pretty well (70% accuracy). Their feature set is rich doesn't seem to include ideology. It was written in 2017; today, I'd expect that a random bright highschooler could do much beter. +- [Forecasting state expenses for budget is always a best guess](https://www.mercurynews.com/2020/05/20/letter-forecasting-state-expenses-for-budget-is-always-a-best-guess/); exactly what it says on the tin. Problem could be solved with a prediction market. +- [From Self-Prediction to Self-Defeat: Behavioral Forecasting, Self-Fulfilling Prophecies, and the Effect of Competitive Expectations](https://pubmed.ncbi.nlm.nih.gov/14561121/). Abstract: Four studies explored behavioral forecasting and the effect of competitive expectations in the context of negotiations. Study 1 examined negotiators' forecasts of how they would behave when faced with a very competitive versus a less competitive opponent and found that negotiators believed they would become more competitive. Studies 2 and 3 examined actual behaviors during a negotiation and found that negotiators who expected a very competitive opponent actually became less competitive, as evidenced by setting lower, less aggressive reservation prices, making less demanding counteroffers, and ultimately agreeing to lower negotiated outcomes. Finally, Study 4 provided a direct test of the disconnection between negotiators' forecasts for their behavior and their actual behaviors within the same sample and found systematic errors in behavioral forecasting as well as evidence for the self-fulfilling effects of possessing a competitive expectation. +- [Neuroimaging results altered by varying analysis pipelines](https://www.nature.com/articles/d41586-020-01282-z). Relevant paragraph: "the authors ran separate ‘prediction markets’, one for the analysis teams and one for researchers who did not participate in the analysis. In them, researchers attempted to predict the outcomes of the scientific analyses and received monetary payouts on the basis of how well they predicted performance. Participants — even researchers who had direct knowledge of the data set — consistently overestimated the likelihood of significant findings". Those who had more knowledge did slightly better, however. +- [Austin Health Official Recommends Cancelling All 2020 Large Events, Despite Unclear Forecasting](https://texasscorecard.com/local/austin-health-official-recommends-cancelling-all-2020-large-events-despite-unclear-forecasting/). Texan article does not consider the perspective that one might want to cancel large events precisely because of the forecasting uncertainty. +- [Misunderstanding Of Coronavirus Predictions Is Eerily Similar To Weather Forecasting](https://www.forbes.com/sites/marshallshepherd/2020/05/22/misunderstanding-of-coronavirus-predictions-is-eerily-similar-to-weather-forecasting/#2f1288467f75), Forbes speculates. + + +## Long content +- [Pan-African Heatwave Health Hazard Forecasting](http://www.walker.ac.uk/research/projects/pan-african-heatwave-health-hazard-forecasting/). "The main aim, is to raise the profile of heatwaves as a hazard on a global scale. Hopefully, the project will add evidence to this sparse research area. It could also provide the basis for a heat early warning system." The project looks to be in its early stages, yet nonetheless interesting. +- [How to evaluate 50% predictions](https://www.lesswrong.com/posts/DAc4iuy4D3EiNBt9B/how-to-evaluate-50-predictions). "I commonly hear (sometimes from very smart people) that 50% predictions are meaningless. I think that this is wrong." +- [Named Distributions as Artifacts](https://blog.cerebralab.com/Named%20Distributions%20as%20Artifacts). On how the named distributions we use (the normal distribution, etc.), were selected for being easy to use in pre-computer eras, rather than on being a good ur-prior on distributions for phenomena in this universe. +- [The fallacy of placing confidence in confidence intervals](https://link.springer.com/article/10.3758/s13423-015-0947-8). On how the folk interpretation of confidence intervals can be misguided, as it conflates: a. the long-run probability, before seeing some data, that a procedure will produce an interval which contains the true value, and b. and the probability that a particular interval contains the true value, after seeing the data. This is in contrast to Bayesian theory, which can use the information in the data to determine what is reasonable to believe, in light of the model assumptions and prior information. I found their example where different confidence procedures produce 50% confidence intervals which are nested inside each other particularly funny. Some quotes: + > Using the theory of confidence intervals and the support of two examples, we have shown that CIs do not have the properties that are often claimed on their behalf. Confidence interval theory was developed to solve a very constrained problem: how can one construct a procedure that produces intervals containing the true parameter a fixed proportion of the time? Claims that confidence intervals yield an index of precision, that the values within them are plausible, and that the confidence coefficient can be read as a measure of certainty that the interval contains the true value, are all fallacies and unjustified by confidence interval theory. + > “I am not at all sure that the ‘confidence’ is not a ‘confidence trick.’ Does it really lead us towards what we need – the chance that in the universe which we are sampling the parameter is within these certain limits? I think it does not. I think we are in the position of knowing that either an improbable event has occurred or the parameter in the population is within the limits. To balance these things we must make an estimate and form a judgment as to the likelihood of the parameter in the universe that is, a prior probability – the very thing that is supposed to be eliminated.” + > The existence of multiple, contradictory long-run probabilities brings back into focus the confusion between what we know before the experiment with what we know after the experiment. For any of these confidence procedures, we know before the experiment that 50 % of future CIs will contain the true value. After observing the results, conditioning on a known property of the data — such as, in this case, the variance of the bubbles — can radically alter our assessment of the probability. + > “You keep using that word. I do not think it means what you think it means.” Íñigo Montoya, The Princess Bride (1987) +- [Psychology of Intelligence Analysis](https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/), courtesy of the American Central Intelligence Agency, seemed interesting, and I read chapters 4, 5 and 14. Sometimes forecasting looks like reinventing intelligence analysis; from that perspective, I've found this reference work useful. Thanks to EA Discord user @Willow for bringing this work to my attention. + - Chapter 4: Strategies for Analytical Judgement. Discusses and compares the strengths and weaknesses of four tactics: situational analysis (inside view), applying theory, comparison with historical situations, and immersing oneself on the data. It then brings up several suboptimal tactics for choosing among hypothesis. + - Chapter 5: When does one need more information, and in what shapes does new information come from? + > Once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of his or her estimates. Additional information does, however, lead the analyst to become more confident in the judgment, to the point of overconfidence. + > Experienced analysts have an imperfect understanding of what information they actually use in making judgments. They are unaware of the extent to which their judgments are determined by a few dominant factors, rather than by the systematic integration of all available information. Analysts actually use much less of the available information than they think they do. + > There is strong experimental evidence, however, that such self-insight is usually faulty. The expert perceives his or her own judgmental process, including the number of different kinds of information taken into account, as being considerably more complex than is in fact the case. Experts overestimate the importance of factors that have only a minor impact on their judgment and underestimate the extent to which their decisions are based on a few major variables. In short, people's mental models are simpler than they think, and the analyst is typically unaware not only of which variables should have the greatest influence, but also which variables actually are having the greatest influence. + - Chapter 14: A Checklist for Analysts. "Traditionally, analysts at all levels devote little attention to improving how they think. To penetrate the heart and soul of the problem of improving analysis, it is necessary to better understand, influence, and guide the mental processes of analysts themselves." The Chapter also contains an Intelligence Analysis reading list. +- [The Limits of Prediction: An Analyst’s Reflections on Forecasting](https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol-63-no-4/Limits-of-Prediction.html), also courtesy of the American Central Intelligence Agency. On how intelligence analysts should inform their users of what they are and aren't capable of. It has some interesting tidbits and references on predicting discontinuities. It also suggests some guiding questions that the analyst may try to answer for the policymaker. +- What is the context and reality of the problem I am facing? +- How does including information on new developments affect my problem/issue? +- What are the ways this situation could play out? +- How do we get from here to there? and/or What should I be looking out for? +> "We do not claim our assessments are infallible. Instead, we assert that we offer our most deeply and objectively based and carefully considered estimates." +- [How to Measure Anything](https://www.lesswrong.com/posts/ybYBCK9D7MZCcdArB/how-to-measure-anything), a review. +- The World Meteorological organization, on their mandate to guarantee that [no one is surprised by a flood](https://public.wmo.int/en/our-mandate/water/no-one-is-surprised-by-a-flood). Browsing the webpage it seems that the organization is either a Key Organization Safeguarding the Vital Interests of the World or Just Another of the Many Bureaucracies Already in Existence, but it's unclear how to differentiate between the two. +- [95%-ile isn't that good](https://danluu.com/p95-skill/): "Reaching 95%-ile isn't very impressive because it's not that hard to do." +- [The Backwards Arrow of Time of the Coherently Bayesian Statistical Mechanic](https://arxiv.org/abs/cond-mat/0410063): +> "Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive inference applied to large mechanical systems. This approach identifies thermodynamic entropy with the information-theoretic uncertainty of an (ideal) observer's subjective distribution over a system's microstates. In this brief note, I show that this postulate, plus the standard Bayesian procedure for updating probabilities, implies that the entropy of a classical system is monotonically non-increasing on the average -- the Bayesian statistical mechanic's arrow of time points backwards. Avoiding this unphysical conclusion requires rejecting the ordinary equations of motion, or practicing an incoherent form of statistical inference, or rejecting the identification of uncertainty and thermodynamic entropy." + +This might be interesting to students in the tradition of E.T. Jaynes: for example, the paper directly conflicts with this LessWrong post: [The Second Law of Thermodynamics, and Engines of Cognition](https://www.lesswrong.com/posts/QkX2bAkwG2EpGvNug/the-second-law-of-thermodynamics-and-engines-of-cognition), part of *Rationality, From AI to Zombies*. The way out might be to postulate that actually, the Bayesian updating process itself would increase entropy, in the form of e.g., the work needed to update bits on a computer. Any applications to Christian lore are left as an excercise for the reader. Otherwise, seeing two bright people being cogently convinced of different perspectives does something funny to my probabilities: it pushes them towards 50%, but also increases the expected time I'd have to spend on the topic to move them away from 50%. +- [Behavioral Problems of Adhering to a Decision Policy](https://pdfs.semanticscholar.org/7a79/28d5f133e4a274dcaec4d0a207daecde8068.pdf) +> Our judges in this study were eight individuals, carefully selected for their expertise as +handicappers. Each judge was presented with a list of 88 variables culled from the past performance charts. He was asked to indicate which five variables out of the 88 he would wish to use when handicapping a race, if all he could have was five variables. He was then asked to indicate which 10, which 20, and which 40 he would use if 10, 20, or 40 were available to him. +> We see that accuracy was as good with five variables as it was with 10, 20, or 40. The flat curve is an average over eight subjects and is somewhat misleading. Three of the eight actually showed a decrease in accuracy with more information, two improved, and three stayed about the same. All of the handicappers became more confident in their judgments as information increased. +The study contains other nuggets, such as: +- An experiment on trying to predict the outcome of a given equation. When the feedback has a margin of error, this confuses respondents. +- "However, the results indicated that subjects often chose one gamble, yet stated a higher selling price for the other gamble" +- "We figured that a comparison between two students along the same dimension should be easier, cognitively, than a 13 comparison between different dimensions, and this ease of use should lead to greater reliance on the common dimension. The data strongly confirmed this hypothesis. Dimensions were weighted more heavily when common than when they were unique attributes. Interrogation of the subjects after the experiment indicated that most did not wish to change their policies by giving more weight to common dimensions and they were unaware that they had done so." +- "The message in these experiments is that the amalgamation of different types of information and different types of values into an overall judgment is a difficult cognitive process. In our attempts to ease the strain of processing information, we often resort to judgmental strategies that do an injustice to the underlying values and policies that we’re trying implement." +- "A major problem that a decision maker faces in his attempt to be faithful to his policy is the fact that his insight into his own behavior may be inaccurate. He may not be aware of the fact that he is employing a different policy than he thinks he’s using. This problem is illustrated by a study that Dan Fleissner, Scott Bauman, and I did, in which 13 stockbrokers and five graduate students served as subjects. Each subject evaluated the potential capital appreciation of 64 securities. [...] A mathematical model was then constructed to predict each subject's judgments. One output from the model was an index of the relative importance of each of the eight information items in determining each subject’s judgments [...] Examination +of Table 4 shows that the broker’s perceived weights did not relate closely to the weights derived from their actual judgments. + +As remedies they suggest to create a model by elliciting the expert, either by having the expert make a large number of judgements and distillating a model, or by asking the expert what they think the most important factors are. A third alternative suggested is computer assistance, so that the experiment participants become aware of which factors influence their judgment. +- [Immanuel Kant, on Betting](https://www.econlib.org/archives/2014/07/kant_on_betting.html) + +Note to the future: All links are added automatically to the Internet Archive. In case of link rot, go [here](https://archive.org/) diff --git a/ea/ForecastingNewsletter/Template.md b/ea/ForecastingNewsletter/Template.md new file mode 100644 index 0000000..9dc335a --- /dev/null +++ b/ea/ForecastingNewsletter/Template.md @@ -0,0 +1,66 @@ +Whatever happened to forecasting? April 2020 +============================================ + +A forecasting digest with a focus on experimental forecasting. You can sign up [here](https://mailchi.mp/18fccca46f83/forecastingnewsletter). The newsletter itself is experimental, but there will be at least five more iterations. + +## Index +- Prediction Markets & Forecasting platforms. + - Augur. + - PredictIt & Election Betting Odds. + - Replication Markets. + - Coronavirus Information Markets. + - Foretold. (c.o.i). + - Metaculus. + - Good Judgement Open. +- In the News. +- Long Content. + +## Prediction Markets & Forecasting platforms. + + + +### Augur: [augur.net](https://www.augur.net/) + +Augur is a decentralized prediction market. + +### Predict It & Election Betting Odds: [predictIt.org](https://www.predictit.org/) & [electionBettingOdds.com](http://electionbettingodds.com/) + +PredictIt is a prediction platform restricted to US citizens or those who bother using a VPN. + +In PredictIt, the [world politics](https://www.predictit.org/markets/5/World) section... + + +[Election Betting Odds](https://electionbettingodds.com/) aggregates PredictIt with other such services for the US presidential elections. + +### Replication Markets: [replicationmarkets.com](https://www.replicationmarkets.com) +Replication Markets is a project where volunteer forecasters try to predict whether a given study's results will be replicated with high power. Rewards are monetary, but only given out to the top N forecasters, and markets suffer from sometimes being dull. + + + +### Coronavirus Information Markets: [coronainformationmarkets.com](https://coronainformationmarkets.com/) +For those who want to put their money where their mouth is, there is now a prediction market for coronavirus related information. The number of questions is small, and the current trading volume started at $8000, but may increase. Another similar platform is [waves.exchange/prediction](https://waves.exchange/prediction), which seems to be just a wallet to which a prediction market has been grafted on. + +Unfortunately, I couldn't make a transaction in these markets with ~30 mins; the time needed to be included in an ethereum block is longer and I may have been too stingy with my gas fee. + +### Foretold: [foretold.io](https://www.foretold.io/) (c.o.i) + +Foretold is an forecasting platform which has experimentation and exploration of forecasting methods in mind. They bring us: + +- A new [distribution builder](https://www.highlyspeculativeestimates.com/dist-builder) to visualize and create probability distributions. + +### Metaculus: [metaculus.com](https://www.metaculus.com/) + +Metaculus is a forecasting platform with an active community and lots of interesting questions. + +### /(Good Judgement?[^]*)|(Superforecast(ing|er))/gi + +Good Judgement Inc. is the organization which grew out of Tetlock's research on forecasting, and out of the Good Judgement Project, which won the [IARPA ACE forecasting competition](https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation_(ACE)_Program), and resulted in the research covered in the *Superforecasting* book. + +Good Judgement Inc. also organizes the Good Judgement Open [gjopen.com](https://www.gjopen.com/), a forecasting platform open to all, with a focus on serious geopolitical questions. They structure their questions in challenges. + +## In the News + +## Grab bag + +## Long Content + diff --git a/index.md~ b/index.md~ deleted file mode 100644 index e720975..0000000 --- a/index.md~ +++ /dev/null @@ -1,122 +0,0 @@ - - -# Welcome / Bienvenidos / Willkommen! - -Welcome to Nuño's lair, with an axiomatic flair for the dramatic, the mathematic, the logical, the philosophical. -I am Nuño Sempere, I am close to the effective altruism and rationality community, and this is where I will keep those projects of mine which I consider worthy. Some of them will be in Spanish, others in German. If you don't understand the title, you probably won't understand the content. - -
- - -
- -## Recent. -[Litany of Light](https://nunosempere.github.io/ea/LitanyOfLight.html) (en) -[Some Data Visualization for Foretold.io's Amplification Experiments](https://observablehq.com/@nunosempere/plots-for-the-amplification-experiment) (en) -[Programming Languages I Know and Cherish](http://nunosempere.github.io/maths-prog/ProgrammingLanguagesIknow) (en) -[A Shapley Value Calculator](http://shapleyvalue.com/) (en) -[On the Recalcitrance of Granularity and Knowledge Integration](https://nunosempere.github.io/rat/BayesRising) - An Answer to Sebastian Benthall's ["Don't Fear the Reaper: Refuting Bostrom's Superintelligence Argument"](https://arxiv.org/abs/1702.08495) (en) -[Why did the Spanish Enlightenment movement fail? (1750-1850)](https://nunosempere.github.io/rat/spanishenlightenment) (en) -[Why did the General Semantics Movement Fail?](https://nunosempere.github.io/rat/general-semantics) (en) -[Discontinuous trends in technological progress](https://nunosempere.github.io/rat/Discontinuous-Progress.html) (en) -[Write-up on some self experimentation in calibration](https://nunosempere.github.io/rat/Self-experimentation-calibration.html) (en) -[Workplace deaths in Bangladesh](https://nunosempere.github.io/rat/workplace-deaths-in-Bangladesh) (en) - -## Effective Altruism & Rationality Stuff / Altruismo efectivo & Racionalidad. - -### EA Forum: -[Myself on the EA Forum](https://forum.effectivealtruism.org/users/nunosempere) (en) -[Shapley values: Better than counterfactuals](https://forum.effectivealtruism.org/posts/XHZJ9i7QBtAJZ6byW/shapley-values-better-than-counterfactuals) (en) -[Why do social movements fail: Two concrete examples](https://forum.effectivealtruism.org/posts/7Pxx7kSQejX2MM2tE/why-do-social-movements-fail-two-concrete-examples) (en) -[EA Mental Health Survey: Results and Analysis.](https://forum.effectivealtruism.org/posts/FheKNFgPqEsN8Nxuv/ea-mental-health-survey-results-and-analysis) (en) - -### Other thoughts: -[Some Data Visualization for Foretold.io's Amplification Experiments](https://observablehq.com/@nunosempere/plots-for-the-amplification-experiment) (en) - -[Write-up on some self experimentation in calibration](https://nunosempere.github.io/rat/Self-experimentation-calibration.html) (en) -[Calibration](https://nunosempere.github.io/calibration/) (en) -[100 predicciones](https://nunosempere.github.io/rat/100-predicciones-en-100-dias.html) (es) -[A practical exercise in p-hacking](https://nunosempere.github.io/rat/eamentalhealth/p-hacking.html) (en) -[List of TAPs](https://nunosempere.github.io/rat/list-of-taps.html) (en) -[List of Superpowers I want to steal](https://nunosempere.github.io/) (en) -[Paradox Party](https://nunosempere.github.io/rat/paradox-party.html) (en) -[Crocker's rules](https://nunosempere.github.io/) (en) -[¿Siempre lo supiste? El sesgo de retrospección](https://nunosempere.github.io/) (es) -[Calibration](https://nunosempere.github.io/https://calibration/) (en) -[From a dark arts practitioner](https://nunosempere.github.io/rat/dark_arts.html) (en) -[Dark Arts: Narrativomancy](https://nunosempere.github.io/miscellanea/narrativomancy.html) (en) -[Consideraciones retreat Altruismo Efectivo Septiembre 2018](https://nunosempere.github.io/rat/AE-retreat.html) (es) -[Mention in *The Times* in a column about the EA Hotel in Blackpool](https://nunosempere.github.io/rat/The-Times.html) (en) -[Different ideals than my own](https://nunosempere.github.io/rat/Different-Ideals.html) - - -## Fucking around with Programming and Maths / Mierdas de programación y matemáticas -[Sci-py turns out to be particularly easy to use](https://nunosempere.github.io/maths-prog/MachineLearningDemystified/) (en, Python) -[Vienna Data Science Hackathon](https://nunosempere.github.io/maths-prog/Vienna-Data-Science-Hackathon-May-4-2019/) (en) -[Letter from O.Teichmüller to Landau](https://nunosempere.github.io/maths-prog/teichmuller.html) (en) -[AI texts I've read](https://nunosempere.github.io/maths-prog/ai.html) (en) -[Sobre el problema de las parejas estables](https://nunosempere.github.io/https://stable-marriage-problem/) (es) -[Mathematicians under the Nazis](https://nunosempere.github.io/projects/mathematicians-under-the-nazis.html) (en) -[Why is the integral which defines the logarithm the inverse of the exponential?](https://nunosempere.github.io/maths-prog/logarithms.pdf) (en) -[Programa de facturación](https://easyfirma.es) (es) - -## Humanities / Humanidades. -[Apuntes de una charla: Diálogos con la cultura, de Remedios Zafra](https://nunosempere.github.io/humanities/remedios-zafra) (es) -[Mentir mola](https://nunosempere.github.io/humanities/mentir-mola.html ) (es) -[¿Qué es atracar un banco en comparación con fundar un banco?: Selección de citas de Brecht.](https://nunosempere.github.io/humanities/brecht.html) (es) -[La desmundanalización de la iglesia](https://nunosempere.github.io/projects/catholic-church.html) (es) - -## Costumbrismo -[Die Macht der Merkel](https://nunosempere.github.io/costumbrismo/merkel.html) (es) -[Oh alemán, mi alemán](https://nunosempere.github.io/costumbrismo/aleman) (es) -[Crónica de una gamberrada](https://nunosempere.github.io/costumbrismo/gamberrada/index.html) (es) - -## Philosophy / Filosofía. -[Monomito y Muchimito: Adiós a los principios, de Odo Marquard](https://nunosempere.github.io/philosophy/marquard.html) (es) -[La nomenclatura no es conocimiento](https://nunosempere.github.io/philosophy/nomenclatura.html) (es) -[Mucha granularidad sí, poca granularidad no](https://nunosempere.github.io/philosophy/granularidad.html) (es) -[Falacia de la subsunción](https://nunosempere.github.io/philosophy/subsuncion.html) (es) -[El concepto griego de verdad & Patologías de la razón](https://nunosempere.github.io/philosophy/aletheia/index.html) (es) -[La paradoja del piano](https://nunosempere.github.io/philosophy/piano.html) (es) -[¿Quién (no) mataría a su profesor de Últimas Tendencias del Arte?](https://nunosempere.github.io/philosophy/arte.html) (es) -[Teorías de la cultura](https://nunosempere.github.io/philosophy/cultura.html) (es) -[El evangelio según San Pedro](https://nunosempere.github.io/philosophy/san-pedro.html) (es) -[El fetichismo del franquismo](https://nunosempere.github.io/philosophy/franquismo.html) (es) - -## Fucking around with Language and Literature / Mierdas lingüísticas. -[Análisis de "Yo persigo una forma que no encuentra mi estilo", de Rubén Darío](https://nunosempere.github.io/lit/ruben-dario-yo-persigo-una-forma-que-no-encuentra-mi-estilo.html) (es) -[Sinónimos de "fantástico" que empiecen por f](https://nunosempere.github.io/lit/fantastico.html) (es) -[En tanto que de rosa y azucena](https://nunosempere.github.io/lit/en-tanto-que-de-rosa-y-azucena.html) (es) -[Englicismos](https://nunosempere.github.io/lit/englicismos) (es) -[Juan Rulfo vs García Márquez](https://nunosempere.github.io/lit/rulfo-garcia.html) (es) -[A buen entendedor, patada en los cojones](https://nunosempere.github.io/lit/patada-en-los-cojones.html) (es) -[Teoría de la literatura](https://nunosempere.github.io/lit/teoria-de-la-literatura.html) (es) -[Las tildes en español](https://nunosempere.github.io/https://tildes/index.html) (es) -[Términos retóricos](https://nunosempere.github.io/lit/terminos-retoricos.html) (es) -[Algunas bonitas palabras en desuso](https://nunosempere.github.io/lit/desuso.html) (es) -[Una interesante tesis doctoral de lingüística](https://nunosempere.github.io/lit/tesis/madurez) (es) -[Todo lo que nunca has querido saber sobre literatura fantástica (mazacote)](https://nunosempere.github.io/lit-fantastica.html) (es) -[Criba palabras](https://nunosempere.github.io/https://criba-de-palabras-Lucia/README.html) (es) - -## Fiction / Ficción -[Las aventuras del aventurero Nuño Núñez](https://nunosempere.github.io/fiction/nuno-nunez.html) (es) -[El Partido Carlista Antimonárquico Español](https://nunosempere.github.io/fiction/carlista.html) (es) - -## Colaboradores con el Régimen. -Claudia Lombardo [Biología para futuros presidentes](https://nunosempere.github.io/) (es) -Lucía Trillo [Thinking fast and slow](https://nunosempere.github.io/) (es) -Alonso Campos [Freedom regained, de Julian Baggini](https://nunosempere.github.io/) (es) - -## Miscellanea / Cajón de sastre. -[Unfair chess](https://nunosempere.github.io/miscellanea/unfairchess.html) (en) -[Made a webpage for a friend](https://ciruelahaiti.github.io/) -[The Wonderer vs Can You Feel the Love Tonight](https://nunosempere.github.io/) (es) -[The Lady of Shalott](https://nunosempere.github.io/) (en) -[Sobre la conquista del fuego, Freud](https://nunosempere.github.io/) (es) -[¿Cuánto vale cada idioma?](https://nunosempere.github.io/) (es) -[Adolfo Suárez desde la tumba](https://nunosempere.github.io/) (es) -[La verdad sobre el caso Amancio Ortega](https://nunosempere.github.io/miscellanea/inditex.html) (es) -[Letimotiv: Ideas recurrentes](https://nunosempere.github.io/miscellanea/letimotiv.html) (es) -[The Mask](https://nunosempere.github.io/miscellanea/The-Mask.html) - -// Note to self: The last thing I added from my other blog was "Patrones en el Economist". diff --git a/test.md b/test.md deleted file mode 100644 index 193a1a8..0000000 --- a/test.md +++ /dev/null @@ -1 +0,0 @@ -testw diff --git a/test.md~ b/test.md~ deleted file mode 100644 index 9daeafb..0000000 --- a/test.md~ +++ /dev/null @@ -1 +0,0 @@ -test