============== world wide web ============== Index: * web browser * search engines * youtube * maps * hacker news, reddit * stack overflow et al * tor web browser =========== Out of all the console web browsers I tested (w3m, lynx, links and elinks) w3m provides the best browsing experience. It has vi key bindings, webpage rendering is intuitive, it can be configured to show pictures, play videos or to download websites and torrents. By default color of the links is blue and it's not that well visible on the console. Go to options (key ) and change it to green, switching it with image link color like that: anchor_color blue -> green image_color green -> blue Default image viewer, w3mimgdisplay, is ok, but it doesn't display gif images and it doesn't allow for zooming the images. You can define other applications to handle that though. File ~/.w3m/mailcap contains information on how to handle different media types. Put these lines there: image/gif; mpv --vo=drm --loop '%s' image/*; fim -a '%s' video/*; mpv --vo=drm --loop '%s' audio/*; mpv '%s' Now when you encounter a gif or an image you can press + to open the link with appropriate program. To check the file type press when cursor is over a link to see the url. There's also another way to open links in other programs - via external browsers. This allows for adding many cool functionalities. You can set them up in options. Each defined external browser can be invoked by pressing (in that order, note the case in m). The main external browser can be invoked without the number. Some ideas for external browsers: - download a webpage under the link wget -pkEe robots=off - play a video (e.g. from invidio.us instance or youtube) mpv --vo drm - view more information on youtube video (description, comments etc.) mpsyt url - add a magnet link to transmission torrent client download list transmission-remote -n user1:password1 -a "$0" - create a torrent file from a magnet link in rtorrent watch location: sh -c 'echo "d10:magnet-uri${#0}:${0}e" > \ /tmp/watch_dir/magnetlink_$RANDOM$RANDOM.torrent' It's also useful to make aliases and functions in your shell to conveniently use search engines directly from command line. Like this: alias w3m='w3m -B' # start w3m on bookmark page w3d() { command w3m duckduckgo.com/lite/?kd=-1\&kp=-2\&kl=us-en\&q="$*"; } w3y() { command w3m youtube.com/results?search_query="$*"; } w3yy() { command w3m youtube.com/results?search_query="$*"; } You can now type '$ w3d atari 65xe' to quickly find search engine results, or '$ w3y 56k modem sounds' to see youtube videos relevant for your phrase. search engines ============== The only major one that works flawlessly with w3m is duckduckgo. Others are working more or less fine, so it's still ok to type for example '!s your query' in duckduckgo (that's for startpage redirection, recommended) to browse other search engines' results. Subpage of duckduckgo that's rendered the best is https://duckduckgo.com/lite, so put this one in a bookmark. Another search engine working great with w3m is search.marginalia.nu. youtube ======= Above you could see how to use youtube from w3m web browser. From the command line you can use youtube-dl to download the videos and watch them with mpv. $ youtube-dl youtube.com/watch?v=vPzDTfIb0DU $ mpv --vo=drm Faith\ No\ More\ -\ Easy*-vPzDTfIb0DU.mkv You can also play youtube videos directly from mpv (don't forget "https://"): $ mpv --vo drm https://youtube.com/watch?v=vPzDTfIb0DU And you can just listen to music without downloading the videos, which is way faster obviously: $ youtube-dl -x youtube.com/watch?v=vPzDTfIb0DU $ mpv Faith\ No\ More\ -\ Easy*-vPzDTfIb0DU.opus $ mpv --no-video https://youtube.com/watch?v=vPzDTfIb0DU To see comments and descriptions for videos you can use mpsyt like this: $ mpsyt > /faith no more # search for phrase > c 3 # comments for search result number 3 > n # next page of comments > i 3 # info and description of the video > 3 # play video > help # show other commands Invidio.us has all youtube videos without all the unnecessary javascript (this doesn't matter for w3m though) and is rendered way better by w3m than original youtube site. youtube-dl fork that works the best currently is github.com/yt-dlp/yt-dlp. maps ==== This is something I need to do some research on. Maybe selenium will allow to use google maps in console? Dunno yet. Now, when I need to see a location of a city I make this query in duckduckgo.com on w3m - `site:freecountrymaps.com berlin`. Far from perfect solution. Or even good tbh. I still fire up xorg to find the best route between map points. hacker news, reddit =================== Hacker news works great in w3m without any customizations. You can't create an account though, as verification is required via google's javascript. Still, you can send an e-mail with a request for an account. All other things work just great, apart from folding the comment threads. Oh, and downvoted comments are not greyed out, so you need to think for yourself to determine if the comment is low quality. Oh, and search doesn't work, but you can use duckduckgo for that with 'site:ycombinator.com tptacek' as an example. Reddit works ok in w3m to view posts. You just have to redirect all http requests to old.reddit.com. To do that create a file ~/.w3m/siteconf file with these lines: url "https://www.reddit.com/" substitute_url "https://old.reddit.com/" I didn't test creating an account, logging in or posting comments, but chances are that this doesn't work well in w3m. You can also try to use programs that show reddit content in terminal, like rtv alternatives[x30]. [x30]:https://github.com/michael-lazar/rtv/blob/master/ALTERNATIVES.md twitter ======= 'twitter.com' can be replaced by javascript-free version 'nitter.eu', while keeping the rest of the url the same. You can also change User-Agent to google bot: 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html'. Twitter.com maintains javascript-free version for crawlers, so w3m renders it just fine with the above User-Agent. stack overflow et al ==================== Logging in from w3m works. Posting questions and answers works if you are not connected via tor or shared VPN (javascript captcha tries to appear otherwise). Posting comments and voting don't work (because javascript is required for that). Creating an API key works. socli[x40] is probably great too, but currently broken[x41] and limited. $ mkdir socli; cd socli $ virtualenv env $ . env/bin/activate $ pip install socli $ socli for loop in python No results found... $ git clone https://github.com/gautamkrishnar/socli # for latest patches $ cd socli $ git checkout refactoring $ pip install -r requirements.txt $ python -m socli for loop in python No results found... [x40]:https://github.com/gautamkrishnar/socli [x41]:https://github.com/gautamkrishnar/socli/issues/163 tor === To visit .onion webpages from w3m you need to install privoxy and tor daemon. In /etc/privoxy/config add these lines to setup a tor proxy server: listen-address 127.0.0.1:8080 forward-socks5 / localhost:9050 . To resolve .onion urls to tor IP adresses you need to put this in /etc/tor/torrc: ## Torified DNS DNSPort 127.0.0.1:53 AutomapHostsOnResolve 1 Also, add this entry at the end of /etc/resolv.conf: nameserver 127.0.0.1 Make sure that all directories specified in torrc exist, otherwise tor will exit without any clear error message. And finally - start tor with 'tor' command and privoxy with 'privoxy --no-daemon --user privoxy /etc/privoxy/config'. Now just open w3m, go to options and set option 'use proxy' to yes. You can now open any .onion site in linux console. WARNING! If you want or need the best anonymity use Tor Browser in Xorg.