From coff at tuhs.org Wed Feb 1 11:50:05 2023 From: coff at tuhs.org (segaloco via COFF) Date: Wed, 01 Feb 2023 01:50:05 +0000 Subject: [COFF] [COFF from TUHS] Re: yet another C discussion (YACD) and: Rust is not C++ In-Reply-To: References: <20230201004023.rHE9j%steffen@sdaoden.eu> <20230201004939.GB6988@mcvoy.com> Message-ID: COFF'd > I think general software engineering knowledge and experience cannot be > 'obsoleted' or made less relevant by better languages. If they help, > great, but you have to do the other part too. As languages advance and > get better at catching (certain kinds of) mistakes, I worry that > engineers are not putting enough time into observation and understanding > of how their programs actually work (or do not). I think you nailed it there mentioning engineers in that one of the growing norms these days is making software development more accessible to a diverse set of backgrounds. No longer does a programming language have to just bridge the gap between, say, an expert mathematician and a compute device. Now there are languages to allow UX designers to declaratively define interfaces, for data scientists to functionally define algorithms, and WYSIWYG editors for all sorts of things that were traditionally handled by hammering out code. The concern of describing a program through a standard language and the concern that language then describing the operations of a specific device have been growing more and more decoupled as time goes on, and that then puts a lot of the responsibility for "correctness" on those creating all these various languages. Whatever concern an engineer originally had to some matter of memory safety, efficiency, concurrency, etc. is now being decided by some team working on the given language of the week, sometimes to great results, other times to disastrous ones. On the flip side, the person consuming the language or components then doesn't need to think about these things, which could go either way. If they're always going to work in this paradigm where they're offloading the concern of memory safety to their language architect of choice, then perhaps they're not shorting themselves any. However, they're then technically not seeing the big picture of what they're working on, which contributes to the diverse quality of software we have today. Long story short, most people don't know how their programs work because they aren't really "their" programs so much as their assembly of a number of off-the-shelf or slightly tweaked components following the norms of whatever school of thought they may originate in (marketing, finance, graphic design, etc.). Sadly, this decoupling likely isn't going away, and we're only bound to see the percentage of "bad" software increase over time. That's the sort of change that over time leads to people then changing their opinions of what "bad software" is. Look at how many people gleefully accept the landscape of smart-device "apps".... - Matt G. From ralph at inputplus.co.uk Wed Feb 1 19:36:31 2023 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Wed, 01 Feb 2023 09:36:31 +0000 Subject: [COFF] Formatting of Expressions. (Was: Re: FD 2) In-Reply-To: References: <202301300750.30U7oQTh013304@freefriends.org> <20230130150219.GD12306@mcvoy.com> <20230130152703.GE12306@mcvoy.com> <20230130154555.GF12306@mcvoy.com> Message-ID: <20230201093631.704E921E5F@orac.inputplus.co.uk> Hi Dave, COFF'd. > > I'll never do if (a==b&&c==d), always if ((a==b)&&(c==d)). > > Indeed; I *always* use parentheses even if they are not necessary (for > my benefit, at least). I find unnecessary parenthesis annoying and clutter which obscures reading. If parenthesis are used only when overriding the default precedence then this beneficially draws attention to the exception. I doubt mandatory parenthesis are used in maths formulas by those that use them in expression. Whitespace is beneficial in both maths formulas and expressions. The squashed expression above will often be spaced more. if (a==b && c==d) if (a == b && c == d) Go's source formatter will vary which operators get spaces to reflect precedence, e.g. https://go.dev/play/p/TU95Oz57GuF shows ‘4*3’ differs. fmt.Println(4 * 3) fmt.Println(5 ^ 4*3) fmt.Println(5 ^ 4*3 + 2/1.9) -- Cheers, Ralph. From brad at anduin.eldar.org Wed Feb 1 23:41:59 2023 From: brad at anduin.eldar.org (Brad Spencer) Date: Wed, 01 Feb 2023 08:41:59 -0500 Subject: [COFF] [COFF from TUHS] Re: yet another C discussion (YACD) and: Rust is not C++ In-Reply-To: (message from segaloco via COFF on Wed, 01 Feb 2023 01:50:05 +0000) Message-ID: segaloco via COFF writes: > COFF'd > [snip] > Long story short, most people don't know how their programs work because they aren't really "their" programs so much as their assembly of a number of off-the-shelf or slightly tweaked components following the norms of whatever school of thought they may originate in (marketing, finance, graphic design, etc.). Sadly, this decoupling likely isn't going away, and we're only bound to see the percentage of "bad" software increase over time. That's the sort of change that over time leads to people then changing their opinions of what "bad software" is. Look at how many people gleefully accept the landscape of smart-device "apps".... > > - Matt G. At my last $DAYJOB the developers were more or less not allowed to alter components that were acquired external to the company. That is to say, no "slightly tweaked" was permitted. If it was in house developed, that was another matter. This led to more then one occasion where a problem that could have been solved with a software fix to the product stack had to be dealt with in infrastructure because they would not fork something they acquired from github. Or they ended up utilizing the infrastructure in a very inefficient manor because they would not alter something or other and then blamed infrastructure for having bad behavior. I am pretty sure that the general understanding of what was being developed was low with most of that group. The development group was intentionally not writing software for the long haul. If something didn't work it was refactored, if possible and if the "it" was important enough, or infrastructure was blamed and made to work around the problem or they just forced the user community to deal with the problems (especially if the user community was in house coworkers). The life cycle of much of the code was less than 3 years and in a lot of cases was reimplemented every year (there were some exceptions, of course...). It may have been "bad software" but as long as it worked for its purpose right now, that really didn't matter. -- Brad Spencer - brad at anduin.eldar.org - KC8VKS - http://anduin.eldar.org From clemc at ccc.com Thu Feb 2 05:02:05 2023 From: clemc at ccc.com (Clem Cole) Date: Wed, 1 Feb 2023 14:02:05 -0500 Subject: [COFF] [TUHS] Re: Unix game origins - stories similar to Crowther's Adventure In-Reply-To: <9rlsK1lcp2ffNj2lkBadLwVDPzGlSjgLbwwByJXMv4FDhSsmezMSUlcXw4jZjc0rM8XtULgrAv6I2g-nurUzaS_fZicDHORQVLSU0fzEydY=@protonmail.com> References: <9rlsK1lcp2ffNj2lkBadLwVDPzGlSjgLbwwByJXMv4FDhSsmezMSUlcXw4jZjc0rM8XtULgrAv6I2g-nurUzaS_fZicDHORQVLSU0fzEydY=@protonmail.com> Message-ID: Switching to COFF On Wed, Feb 1, 2023 at 1:33 PM segaloco via TUHS wrote: > In the annals of UNIX gaming, have there ever been notable games that have > operated as multiple processes, perhaps using formal IPC or even just pipes > or shared files for communication between separate processes (games with > networking notwithstanding)? > Yes - there were a number of them. Both for UNIX and other wise. Some spanned the Arpanet back in the day on the PDP-10's. There was an early first person shooter games that I remember that ran on the PDP-10s on ADM3As and VT52 that worked that way. You flew into space and fought each other. CMU's (Steve Rubin's) Trip was stand alone program - sort of the grand-daddy of the Star Trek games. It ran on a GDP2 (Triple-Drip Graphics Wonder) and had dedicated 11/20. It was multiple processes to do everything. You were at the Captions chair of the Enterprise looking out into space. You had various mission and at some point would bee to reprovision - which meant you had to dock at the 2001 space station including timing your rotation to line up with docking bay like in the movie. When you beat an alien ship you got a bottle of coke - all of which collected in row on the bottom of the screen. I did manage to save the (BLISS-11) sources to it a few years ago. One of my dreams is to try to write GDP simulator for SIMH and see if we can bring it back to life. A big issue as Rob knows is the GDPs had an amazing keyboard so duplicating it will take some thinking with modern HW; but HW has caught up such that I think it might be possible to emulate it. SIMH works really well with a number of the other Graphics systems and with my modem system like my current Mac and its graphics HW, there might be a chance. One of my other favorites was one that ran on the Xerox Alto's who's name I don't remember, where you wandered around the Xerox 3M ethernet. People would enter your system and appear on your system. IIRC Byte Magazine did an article that talked about it at one point -- this was all pre-Apple Macs - but I remember they had pictures of people playing it that I think they took at Stanford. IIRC Shortly after the X-Terminals appeared somebody tried to duplicate it, or maybe that was with the Bilts but it was not quite as good as those of us that had access to real Xerox Altos. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Thu Feb 2 05:16:33 2023 From: crossd at gmail.com (Dan Cross) Date: Wed, 1 Feb 2023 14:16:33 -0500 Subject: [COFF] [TUHS] Re: Unix game origins - stories similar to Crowther's Adventure In-Reply-To: References: <9rlsK1lcp2ffNj2lkBadLwVDPzGlSjgLbwwByJXMv4FDhSsmezMSUlcXw4jZjc0rM8XtULgrAv6I2g-nurUzaS_fZicDHORQVLSU0fzEydY=@protonmail.com> Message-ID: [TUHS to Bcc] On Wed, Feb 1, 2023 at 2:11 PM Rich Salz wrote: > On Wed, Feb 1, 2023 at 1:33 PM segaloco via TUHS wrote: >> In the annals of UNIX gaming, have there ever been notable games that have operated as multiple processes, perhaps using formal IPC or even just pipes or shared files for communication between separate processes (games with networking notwithstanding)? > > https://www.unix.com/man-page/bsd/6/hunt/ > source at http://ftp.funet.fi/pub/unix/4.3bsd/reno/games/hunt/hunt/ Hunt was the one that I thought of immediately. We used to play that on Suns and VAXen and it could be lively. There were a number of such games, as Clem mentioned; others I remember were xtrek, hearts, and various Chess and Go servers. - Dan C. From steffen at sdaoden.eu Thu Feb 2 05:29:38 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Wed, 01 Feb 2023 20:29:38 +0100 Subject: [COFF] [TUHS] Algol rules: was something about Rust is not C++ In-Reply-To: <3808a35c-2ee0-2081-4128-c8196b4732c0@gmail.com> References: <20230201004023.rHE9j%steffen@sdaoden.eu> <3808a35c-2ee0-2081-4128-c8196b4732c0@gmail.com> Message-ID: <20230201192938.ozV2d%steffen@sdaoden.eu> Will Senn wrote in <3808a35c-2ee0-2081-4128-c8196b4732c0 at gmail.com>: |Well, I just read this as Rust is dead... here's hoping, but seriously, |if we're gonna go off and have a language vs language discussion, I |personally don't think we've had any real language innovation since |Algol 60, well maybe lisp... sheesh, surely we're in COFF territory. It has evangelists on all fronts. ..Yes it was only that while i was writing the message i reread about Vala of the GNOME project, which seems to be a modern language with many beneficial properties still, growing out of a Swiss University (is that a bad sign to come from Switzerland and more out of research), and it had support of Ubuntu and many other parts of the GNOME project. Still it is said to be dead. I scrubbed that part of my message. But maybe thus a "dead" relation in between the lines remained. Smalltalk is also such a thing, though not from Switzerland. An Ach! on the mystery of human behaviour. Or, like the wonderful Marcel Reich-Ranicki said, "Wir sehen es betroffen, den Vorhang zu, und alle Fragen offen" ("Concerned we see, the curtain closed, and all the Questions open"). --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From steffen at sdaoden.eu Thu Feb 2 06:00:42 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Wed, 01 Feb 2023 21:00:42 +0100 Subject: [COFF] [TUHS] Algol rules: was something about Rust is not C++ In-Reply-To: <20230201192938.ozV2d%steffen@sdaoden.eu> References: <20230201004023.rHE9j%steffen@sdaoden.eu> <3808a35c-2ee0-2081-4128-c8196b4732c0@gmail.com> <20230201192938.ozV2d%steffen@sdaoden.eu> Message-ID: <20230201200042.D6ytp%steffen@sdaoden.eu> Steffen Nurpmeso wrote in <20230201192938.ozV2d%steffen at sdaoden.eu>: |Will Senn wrote in | <3808a35c-2ee0-2081-4128-c8196b4732c0 at gmail.com>: ||Well, I just read this as Rust is dead... here's hoping, but seriously, ... Talking about dead, i suddenly had the braindead desire to reformat all my actively maintained things (sh,perl,python,C) in the Christian Christmas time. Quite sudden i was looking at the about 30 percent of my laptop screen that i actually use (COLUMNS=191 LINES=55, but that only after libxft-2.3.7 introduced a bug that is still not fixed, it was more more before (i was using Xft.embolden with LiberationMono which i had forgotten, but otherwise with Xft.antialias it looks terrible still .. with that many lines)), and was thinking "widescreen will never disappear again". Thus i dropped my yet always used attitude of 79 columns no matter what, went back to tabulator indentation, and now use 120 columns, without being strict (but max of about 140, as rarely as possible). (Some old legacy sources still use the old style, but anything planned to remain was converted.) If that is not stupid and brain dead, i do not know. Welcome back tabulators (it saved a lot of bytes!), and welcome new widescreen world. I do not regret it. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From crossd at gmail.com Thu Feb 2 09:24:18 2023 From: crossd at gmail.com (Dan Cross) Date: Wed, 1 Feb 2023 18:24:18 -0500 Subject: [COFF] [TUHS] Re: Unix game origins - stories similar to Crowther's Adventure In-Reply-To: References: Message-ID: [TUHS to Bcc] On Wed, Feb 1, 2023 at 3:23 PM Douglas McIlroy wrote: > > In the annals of UNIX gaming, have there ever been notable games that have operated as multiple processes, perhaps using formal IPC or even just pipes or shared files for communication between separate processes > > I don't know any Unix examples, but DTSS (Dartmouth Time Sharing > System) "communication files" were used for the purpose. For a fuller > story see https://www.cs.dartmouth.edu/~doug/DTSS/commfiles.pdf Interesting. This is now being discussed on the Multicians list (which had a DTSS emulator! Done for use by SIPB). Warren Montgomery discussed communication files under DTSS for precisely this kind of thing; apparently he had a chess program he may have run under them. Barry Margolin responded that he wrote a multiuser chat program using them on the DTSS system at Grumman. Margolin suggests a modern Unix-ish analogue may be pseudo-ttys, which came up here earlier (I responded pointing to your wonderful note linked above). > > This is probably a bit more Plan 9-ish than UNIX-ish > > So it was with communication files, which allowed IO system calls to > be handled in userland. Unfortunately, communication files were > complicated and turned out to be an evolutionary dead end. They had > had no ancestral connection to successors like pipes and Plan 9. > Equally unfortunately, 9P, the very foundation of Plan 9, seems to > have met the same fate. I wonder if there was an analogy to multiplexed files, which I admit to knowing very little about. A cursory glance at mpx(2) on 7th Edition at least suggests some surface similarities. - Dan C. From dave at horsfall.org Fri Feb 3 06:37:34 2023 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 3 Feb 2023 07:37:34 +1100 (EST) Subject: [COFF] Early programs that were run more than once? Message-ID: That was the title of a sidebar in Australia's "Silicon Chip" electronics magazine this month, and referred to the alleged practice of running scientific and engineering programs many times to ensure consistent output, as hardware error checks weren't the best in those days (bit-flips due to electrical noise etc). Anyway, the mag is seeking corroboration on this (credit given where due, of course); I find it a bit hard to believe that machines capable of running complex programs did not have e.g. parity checking... Thanks. -- Dave From will.senn at gmail.com Sat Feb 4 01:27:02 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 3 Feb 2023 09:27:02 -0600 Subject: [COFF] converting lousy scans of pdfs into something more useable Message-ID: All, I thought I would post something here that wasn't DOA over on tuhs and see if it would fly here instead. I have been treating coff as the destination for the place where off-topic tuhs posts go to die, but after the latest thread bemoaning a place to go for topics tangential to unix, I thought I'd actually start a coff thread! Here goes... I read a tremendous number of documents from the web, or at least read parts of them - to the tune of maybe 50 or so a week. It is appalling to me in this era that we can't get better at scanning. Be that as it may, the needle doesn't seem to have moved appreciably in the last decade or so and it's a little sad. Sure, if folks print to pdf, it's great. But, if they scan a doc, not so great, even today. Rather than worry about the scanning aspects, I am more interested in what to do with those scans. Can they be handled in such a way as to give them new life? Unlike the scanning side of things, I have found quite a bit of movement in the area of being able to work with the pdfs and I'd really like to get way better at it. If I get a bad scanned pdf, if I can make it legible on screen, legible on print, and searchable, I'm golden. Sadly, that's way harder than it sounds, or, in my opinion, than it should be. I recently put together a workflow that is tenable, if time consuming. If your interested in the details, I've shared them: https://decuser.github.io/pdfs/2023/02/01/pdf-cleanup-workflow.html In the note, I leverage a lot of great tools that have significantly improved over the years to the point where they do a great job at what they do. But, there's lots of room for improvement. Particularly in the area of image tweaking around color and highlights and such. The note is mac-centric in that I use a mac, otherwise, all of the tools work on modern *nix and with a little abstract thought, windows too. In my world, here's what happens: * find a really interesting topic and along the way, collect pdfs to read * open the pdf and find it salient, but not so readable, with sad printability, and no or broken OCR * I begin the process of making the pdf better with the aforementioned goals aforethought The process in a nutshell: 1. Extract the images to individual tiffs (so many tools can't work with multi-image tiffs)     * pdfimages from poppler works great for this 2. Adjust the color (it seems impossible to do this without a batch capable gui app)     * I use Photoscape X for this - just click batch and make adjustments to all of the images using the same settings 3. Resize the images - most pdfs have super wonky sizes     * I use convert from imagemagick for this and I compress the tiffs while I'm converting them 4. Recombine the images into a multi-tiff image     * I use tiffcp from libtiff for this 5. OCR the reworked image set     * I use tesseract for this - It's gotten so much better it's ridiculous This process results in a pdf that meets the objectives. It's not horribly difficult to do and it's not horribly time consuming. It represents many, many attempts to figure out this thorny problem. I'd really like to get away from needing Photoscape X, though. Then I could entirely automate the workflow in bash... The problem is that the image adjustments are the most critical - image extraction, resize, compression, recombining images, ocr (I still can't believe it), and outputting a pdf are now taken care of by command line tools that work well. I wouldn't mind using a gui to figure out some color setting (Grayscale, Black and White, or Color) and increase/decrease values for shadows and highlights if those could then be mapped to command line arguments of a tool that could apply them, though. Cuz, then the workflow could be, extract a good representative page as image, open it, figure out the color settings, and then use those settings with toolY as part of the scripted workflow. Here are the objectives for easy reference: 1. The PDF needs to be readable on a decent monitor (zooming in doesn't distort the readability, pixelation that is systematic is ok, but not preferred). Yes, I know it's got a degree of subjectivity, but blobby, bleeding text is out of scope! 2. The PDF needs to print with a minimum of artifact (weird shadows, bleeding and blob are out). It needs to be easy to read. 3. The PDF needs to be searchable with good accuracy (generally, bad scans have no ocr, or ocr that doesn't work). Size is a consideration, but depends greatly on the value of the work. My own calculus goes like this - if it's modern work, it should be way under 30mbs. If it's print to pdf, it should be way under 10mb (remember when you thought you'd never use 10mb of space... for all of your files and the os). If it is significant and rare, less than 150mbs can work. Obviously, this is totally subjective, your calculus is probably quite different. The reason this isn't posted over in pdf.scans.discussion is that even if there were such a place, it'd be filled with super technical gibberish about color depth and the perils of gamma radiation or somesuch. We, as folks interested in preserving the past have a more pragmatic need for a workable solution that is attainable to mortals. So, with that as a bit of background, let me ask what I asked previously in a different wayon tuhs, here in coff - what's your experience with using sad pdfs? Do you just live with them as they are, or do you try to fix them and how, or do you use a workflow and get good results? Later, Will -------------- next part -------------- An HTML attachment was scrubbed... URL: From drb at msu.edu Sat Feb 4 02:00:10 2023 From: drb at msu.edu (Dennis Boone) Date: Fri, 03 Feb 2023 11:00:10 -0500 Subject: [COFF] converting lousy scans of pdfs into something more useable In-Reply-To: (Your message of Fri, 03 Feb 2023 09:27:02 -0600.) References: Message-ID: <20230203160010.A78264BBB5@yagi.h-net.msu.edu> > I read a tremendous number of documents from the web, or at least > read parts of them - to the tune of maybe 50 or so a week. It is > appalling to me in this era that we can't get better at scanning. Be > that as it may, the needle doesn't seem to have moved appreciably in > the last decade or so and it's a little sad. Sure, if folks print to > pdf, it's great. But, if they scan a doc, not so great, even today. I see a fair number of frustrating scanned-doc PDFs too. My thoughts on what constitutes a decent scan: * Assume people will print at least a few pages occasionally. It's often easier to print that one table or diagram and take it to the bench than to try to use a tablet or run back and forth to a PC. That affects how you think about creating the PDF. * Don't use JPEG 2000 and similar compression algorithms that try to re-use blocks of pixels from elsewhere in the document -- too many errors, and they're errors of the sort that can be critical. Even if the replacements use the correct code point, they're distracting as hell in a different font, size, etc. * OCR-under is good. I use `ocrmypdf`, which uses the Tesseract engine. * I do get angry when I see people trying to reconstruct the document via OCR and omitting the actual scan -- too many errors. * Bookmarks for pages / table of contents entries / etc are mandatory. Very few things make a scanned-doc PDF less useful than not being able to skip directly to a document indicated page. * I like to see at least 300 dpi. * Don't scan in color mode if the source material isn't color. Grey scale or even "line art" works fine in most cases. Using one pixel means you can use G4 compression for colorless pages. * Do reduce the color depth of pages that do contain color if you can. The resulting PDF can contain a mix of image types. I've worked with documents that did use color where four or eight colors were enough, and the whole document could be mapped to them. With care, you _can_ force the scans down to two or three bits per pixel. * Do insert sensible metadata. * Do try to square up the inevitably crooked scans, clean up major floobydust and whatever crud around the edges isn't part of the paper, etc. Besides making the result more readable, it'll help the OCR. I never have any luck with automated page orientation tooling for some reason, so end up just doing this with Gimp. Tuppence. De From bakul at iitbombay.org Sat Feb 4 02:01:37 2023 From: bakul at iitbombay.org (Bakul Shah) Date: Fri, 3 Feb 2023 08:01:37 -0800 Subject: [COFF] converting lousy scans of pdfs into something more useable In-Reply-To: References: Message-ID: <8B9903EF-9F35-4810-85DD-F8629EC67973@iitbombay.org> On Feb 3, 2023, at 7:27 AM, Will Senn wrote: > > what's your experience with using sad pdfs? Do you just live with them as they are, or do you try to fix them and how, or do you use a workflow and get good results? Usually I just live with them but I may use "ocrmypdf" if search or copy-paste is unsatisfactory. https://github.com/ocrmypdf/OCRmyPDF It's a python script that runs most any unix and uses tesseract. Its author's motivation seems similar to yours: I searched the web for a free command line tool to OCR PDF files: I found many, but none of them were really satisfying: • Either they produced PDF files with misplaced text under the image (making copy/paste impossible) • Or they did not handle accents and multilingual characters • Or they changed the resolution of the embedded images • Or they generated ridiculously large PDF files • Or they crashed when trying to OCR • Or they did not produce valid PDF files • On top of that none of them produced PDF/A files (format dedicated for long time storage) ...so I decided to develop my own tool. I rarely print PDFs any more. From will.senn at gmail.com Sat Feb 4 02:21:31 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 3 Feb 2023 10:21:31 -0600 Subject: [COFF] converting lousy scans of pdfs into something more, useable In-Reply-To: <167544017712.2485736.11108085155717490044@minnie.tuhs.org> References: <167544017712.2485736.11108085155717490044@minnie.tuhs.org> Message-ID: > From: Dennis Boone > > * Don't use JPEG 2000 and similar compression algorithms that try to > re-use blocks of pixels from elsewhere in the document -- too many > errors, and they're errors of the sort that can be critical. Even if > the replacements use the correct code point, they're distracting as > hell in a different font, size, etc. I wondered about why certain images were the way they were, this probably explains a lot. > * OCR-under is good. I use `ocrmypdf`, which uses the Tesseract engine. Thanks for the tips. > * Bookmarks for pages / table of contents entries / etc are mandatory. > Very few things make a scanned-doc PDF less useful than not being able > to skip directly to a document indicated page. I wish. This is a tough one. I generally sacrifice ditching the bookmarks to make a better pdf. I need to look into extracting bookmarks and if they can be re-added without getting all wonky. > * I like to see at least 300 dpi. Yes, me too, but I've found that this often results in too big (when fixing existing), if I'm creating, they're fine. > * Don't scan in color mode if the source material isn't color. Grey > scale or even "line art" works fine in most cases. Using one pixel > means you can use G4 compression for colorless pages. Amen :). > > * Do reduce the color depth of pages that do contain color if you can. > The resulting PDF can contain a mix of image types. I've worked with > documents that did use color where four or eight colors were enough, > and the whole document could be mapped to them. With care, you _can_ > force the scans down to two or three bits per pixel. > * Do insert sensible metadata. > > * Do try to square up the inevitably crooked scans, clean up major > floobydust and whatever crud around the edges isn't part of the paper, > etc. Besides making the result more readable, it'll help the OCR. I > never have any luck with automated page orientation tooling for some > reason, so end up just doing this with Gimp. Great points. Thanks. -will From will.senn at gmail.com Sat Feb 4 02:25:50 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 3 Feb 2023 10:25:50 -0600 Subject: [COFF] converting lousy scans of pdfs into something more useable In-Reply-To: <8B9903EF-9F35-4810-85DD-F8629EC67973@iitbombay.org> References: <8B9903EF-9F35-4810-85DD-F8629EC67973@iitbombay.org> Message-ID: On 2/3/23 10:01 AM, Bakul Shah wrote: > > https://github.com/ocrmypdf/OCRmyPDF > > It's a python script that runs most any unix and uses > tesseract. Its author's motivation seems similar to yours: > > I searched the web for a free command line tool to OCR PDF files: I found many, but none of them were really satisfying: > • Either they produced PDF files with misplaced text under the image (making copy/paste impossible) > • Or they did not handle accents and multilingual characters > • Or they changed the resolution of the embedded images > • Or they generated ridiculously large PDF files > • Or they crashed when trying to OCR > • Or they did not produce valid PDF files > • On top of that none of them produced PDF/A files (format dedicated for long time storage) > ...so I decided to develop my own tool. Nice. Off to checking out OCRmyPDF! > I rarely print PDFs any more. I can't seem to get away from having to highlight and mark up the stuff I read. I love pdf's searchability of words, but not for quickly locating a section, or just browsing and studying them. I can flip pages much faster with paper than an ebook it seems :). -will -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sat Feb 4 03:09:13 2023 From: bakul at iitbombay.org (Bakul Shah) Date: Fri, 3 Feb 2023 09:09:13 -0800 Subject: [COFF] converting lousy scans of pdfs into something more useable Message-ID: On Feb 3, 2023, at 8:26 AM, Will Senn wrote: > > I can't seem to get away from having to highlight and mark up the stuff I read. I love pdf's searchability of words, but not for quickly locating a section, or just browsing and studying them. I can flip pages much faster with paper than an ebook it seems :). You can annotate, highlight and markup pdfs. There are apps for that though I’m not very familiar with them as I don’t markup even paper copies. On an iPad you can easily annotate pdfs with an apple pencil. From will.senn at gmail.com Sat Feb 4 03:31:44 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 3 Feb 2023 11:31:44 -0600 Subject: [COFF] move to COFF Re: [TUHS] Re: Proper use of TUHS (was Re: Typesetter C compiler) In-Reply-To: <057be760-5f24-6274-c4b4-97ee8c439615@gmail.com> References: <20230202190232.C79D118C073@mercury.lcs.mit.edu> <73d0b98b-ae3f-428d-1b2e-feada7fc98a1@case.edu> <20230203141540.GS30555@mcvoy.com> <20230203165420.GB19745@mcvoy.com> <057be760-5f24-6274-c4b4-97ee8c439615@gmail.com> Message-ID: <5000bf66-4ae0-710f-c613-3bed61fb5f91@gmail.com> Oh, and of course I would cc the old address! Reply on the correct COFF address Sheesh. On 2/3/23 11:26 AM, Will Senn wrote: > We're in COFF territory again. I am enjoying the conversation, but > let's self monitor. Perhaps, a workflow for this is that when we drift > off into non-unix history discussion, we cc: COFF and tell folks to > continue there? As a test I cced it on this email, don't reply all to > this list. Just let's talk about it over in coff. If you aren't on > coff join it. > > If you aren't sure or think most folks on the list want to discuss it. > Post it on COFF, if you don't get any traction, reference the COFF > thread and tease it in TUHS. > > This isn't at all a gripe - I heart all of our discussions, but I > agree that it's hard to keep it history related here with no outlet > for tangential discussion - so, let's put coff to good use and try it > for those related, but not quite discussions. > > Remember, don't reply to TUHS on this email :)! > > - will > > On 2/3/23 11:11 AM, Steve Nickolas wrote: >> On Fri, 3 Feb 2023, Larry McVoy wrote: >> >>> Some things will never go away, like keep your fingers off of my L1 >>> cache lines.  I think it's mostly lost because of huge memories, but >>> one of the things I love about early Unix is how small everything was. >>> Most people don't care, but if you want to go really fast, there is no >>> replacement for small. >>> >>> Personally, I'm fine with some amount of "list about new systems where >>> we can ask about history because that helps us build those new >>> systems". >>> Might be just me, I love systems discussions. >> >> I find a lot of my own stuff is like this - kindasorta fits and >> kindasorta doesn't for similar reasons. >> >> (Since a lot of what I've been doing lately is creating a >> SysV-flavored rewrite of Unix from my own perspective as a >> 40-something who actually got most of my experience coding for >> 16-bits and MS-DOS, and speaks fluent but non-native C.  I'm sure it >> comes out in my coding style.) >> >> -uso. > From ralph at inputplus.co.uk Sat Feb 4 17:59:16 2023 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sat, 04 Feb 2023 07:59:16 +0000 Subject: [COFF] converting lousy scans of pdfs into something more useable In-Reply-To: References: Message-ID: <20230204075916.C9C8621C9A@orac.inputplus.co.uk> > https://decuser.github.io/pdfs/2023/02/01/pdf-cleanup-workflow.html units(1) can do the sum for you. $ units -1 475point in * 6.5972222 Be aware there are several kinds of ‘point’; search /usr/share/units/definitions.units for /^point to get to the relevant area. It will also do your simple ‘echo ... | bc -l’ sums. $ units 2560/96 Definition: 26.666667 GraphicsMagick, gm(1), may be more consistent and future-proof than ImageMagick. > I'd really like to get away from needing Photoscape X, though. > Then I could entirely automate the workflow in bash... gmic(1) is a very powerful and little known image-processing program. It may help dump PhotoScape X. -- Cheers, Ralph. From rtomek at ceti.pl Sun Feb 5 08:38:26 2023 From: rtomek at ceti.pl (Tomasz Rola) Date: Sat, 4 Feb 2023 23:38:26 +0100 Subject: [COFF] [TUHS] Re: Proper use of TUHS (was Re: Typesetter C compiler) In-Reply-To: <7wv8kj6yrx.fsf@junk.nocrew.org> References: <20230202190232.C79D118C073@mercury.lcs.mit.edu> <7wv8kj6yrx.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 03, 2023 at 06:36:34AM +0000, Lars Brinkhoff wrote: > Dan Cross wrote: > > So, the question becomes: what _is_ that forum, if such a thing > > exists at all? > > Some options: > > - Cctalk email list. (cc-ed to coff, of coffse...) I use to hang out on IBM-MAIN mailing list, too. While they are, mostly, dealing with modern mainframes and current problems, they also occasionally mention old story or two. Actually, since mainframe is such a living fossil thing, the whole talk sometimes feels as if it was about something upgraded continuously from the 1960-ties. Most of it is uncomprehensible to me (never had proper mainframe training, or unproper one, and they deal with stuff in unique way, have their own acronyms for things, there are some intro books but there is not enough time*energy), but also a bit educating - a bit today, a bit next week etc. > - ClassicCMP Discord. > - Retrocomputingforum.com. > - Various Facebook groups. Web stuff, requiring Javascript to work, ugh, ugh-oh. Mostly, it boils down to the fact that one cannot easily curl the text from those other places (AFAICT). So it is hard to awk this text into mbox format and read it comfortably. -- Regards, Tomasz Rola -- ** A C programmer asked whether computer had Buddha's nature. ** ** As the answer, master did "rm -rif" on the programmer's home ** ** directory. And then the C programmer became enlightened... ** ** ** ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** From coff at tuhs.org Tue Feb 7 03:37:38 2023 From: coff at tuhs.org (segaloco via COFF) Date: Mon, 06 Feb 2023 17:37:38 +0000 Subject: [COFF] VAX/VMS 4.4 Source Listing? Message-ID: Good morning all, I was wondering if anyone in this group was aware of any known preservation of VAX/VMS 4.4 source code? Just saw this on eBay: https://www.ebay.com/itm/195582389147?hash=item2d899e6b9b:g:neYAAOSwmQJj3EkH I certainly don't have the equipment for this in my arsenal, but at the same time, if this represents long-lost source code, I'd happy try and nab it and then get it to someone who can do the restoration work from this. Thoughts? - Matt G. From clemc at ccc.com Tue Feb 7 06:23:54 2023 From: clemc at ccc.com (Clem Cole) Date: Mon, 6 Feb 2023 15:23:54 -0500 Subject: [COFF] VAX/VMS 4.4 Source Listing? In-Reply-To: References: Message-ID: Matt, I sent a former co-worker the VMS developer lead an email this AM. I was pretty sure I knew the answer to my question when I asked if this had any value. His reply was: *Not sure about this particular version, but there certainly are other copies floating around.* *One thing to be aware of is that to my knowledge**, DEC never released a complete source kit outside of the company at any price *(... we certainly had that behavior in Ultrix/Tru64-land I know for sure...)*. The kits redacted "sensitive" information, including some of the authentication methods and the license enforcement (er, compliance assistance) code. What was sold as a "source kit" (media, under license) was, nonetheless considered a trade secret with all the markings. * *I had a complete copy of the master source pack in Marlborough during the Aquarius development/debug time; I know I put backup tape of that into the CHM. That has the CMS library which would go back to at least V1. (That would be c.a. 1987-9.) But the master pack contains only the VMS OS & some tightly coupled products (e.g. rms). The languages were elsewhere, as was the workstation software.* Clem ᐧ On Mon, Feb 6, 2023 at 12:38 PM segaloco via COFF wrote: > Good morning all, I was wondering if anyone in this group was aware of any > known preservation of VAX/VMS 4.4 source code? > > Just saw this on eBay: > https://www.ebay.com/itm/195582389147?hash=item2d899e6b9b:g:neYAAOSwmQJj3EkH > > I certainly don't have the equipment for this in my arsenal, but at the > same time, if this represents long-lost source code, I'd happy try and nab > it and then get it to someone who can do the restoration work from this. > > Thoughts? > > - Matt G. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Tue Feb 7 06:36:48 2023 From: imp at bsdimp.com (Warner Losh) Date: Mon, 6 Feb 2023 13:36:48 -0700 Subject: [COFF] VAX/VMS 4.4 Source Listing? In-Reply-To: References: Message-ID: On Mon, Feb 6, 2023, 1:25 PM Clem Cole wrote: > Matt, > I sent a former co-worker the VMS developer lead an email this AM. I > was pretty sure I knew the answer to my question when I asked if this had > any value. His reply was: > > *Not sure about this particular version, but there certainly are other > copies floating around.* > > *One thing to be aware of is that to my knowledge**, DEC never released a > complete source kit outside of the company at any price *(... we > certainly had that behavior in Ultrix/Tru64-land I know for sure...)*. > The kits redacted "sensitive" information, including some of the > authentication methods and the license enforcement (er, compliance > assistance) code. What was sold as a "source kit" (media, under license) > was, nonetheless considered a trade secret with all the markings. * > > *I had a complete copy of the master source pack in Marlborough during the > Aquarius development/debug time; I know I put backup tape of that into the > CHM. That has the CMS library which would go back to at least V1. (That > would be c.a. 1987-9.) But the master* > > V1 is 77-79... or is the CMS from that time? Warner * pack contains only the VMS OS & some tightly coupled products (e.g. > rms). The languages were elsewhere, as was the workstation software.* > > > Clem > ᐧ > > On Mon, Feb 6, 2023 at 12:38 PM segaloco via COFF wrote: > >> Good morning all, I was wondering if anyone in this group was aware of >> any known preservation of VAX/VMS 4.4 source code? >> >> Just saw this on eBay: >> https://www.ebay.com/itm/195582389147?hash=item2d899e6b9b:g:neYAAOSwmQJj3EkH >> >> I certainly don't have the equipment for this in my arsenal, but at the >> same time, if this represents long-lost source code, I'd happy try and nab >> it and then get it to someone who can do the restoration work from this. >> >> Thoughts? >> >> - Matt G. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Wed Feb 8 02:40:36 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 7 Feb 2023 11:40:36 -0500 Subject: [COFF] VAX/VMS 4.4 Source Listing? In-Reply-To: References: Message-ID: On 2/6/23, Warner Losh wrote: > On Mon, Feb 6, 2023, 1:25 PM Clem Cole wrote: [quoting a VMS developer friend of his] >> >> *I had a complete copy of the master source pack in Marlborough during >> the >> Aquarius development/debug time; I know I put backup tape of that into >> the >> CHM. That has the CMS library which would go back to at least V1. (That >> would be c.a. 1987-9.) But the master* >> >> V1 is 77-79... or is the CMS from that time? I worked at one of the VAX-11/780 hardware beta test sites back in 1977-79. The machine ran VAX/VMS verison X0.5 at first, later upgraded to V1.0. I joined DEC in 1980 in the Methods & Tools software engineering group. At that time they were working on a tool called the System for Tracking the Evolution of Programs (STEP). This was a version control system using ideas from Unix's SCCS. The name STEP had trademark issues and so it was rechristened Code Management System (CMS). I think it was first released by DEC as a product in 1982. STEP would have been in early stages of development during the time that VMS 1.0 was being developed. The VMS group at that time used a tool called SLP as the core of their configuration management system. SLP could compare two versions of a text file and create a patch file (these were called "slipper packets") to convert one into the other. Configuration management consisted of keeping collections of slipper packets. The VMS development group converted to CMS ca. 82/83. I don't know which version of VMS was the base version for their CMS library. They probably started with the archived V1.0 release sources. But their CMS library would not have recorded the development history of V1.0. -Paul W. From crossd at gmail.com Thu Feb 9 07:28:41 2023 From: crossd at gmail.com (Dan Cross) Date: Wed, 8 Feb 2023 16:28:41 -0500 Subject: [COFF] [TUHS] Re: history of community help for unix users everywhere In-Reply-To: References: Message-ID: [TUHS to Bcc: and +COFF] On Wed, Feb 8, 2023 at 3:50 PM Warner Losh wrote: > [snip] > The community aspect of open source was there in spades as well, with people helping other people and sharing fixes. But it was complicated by restrictive license agreements and somewhat (imho) overzealous protection of 'rights' at times that hampered things and would have echos in later open source licenses and attitudes that would develop in response. Even though the term 'open source' wasn't coined until 1998, the open source ethos were present in many of the early computer users groups, not least the unix ones. Don't forget SHARE! Honestly, I think the IBM mainframe community doesn't get its due. There was actually a lot of good stuff there. > USENET amplified it, plus let in the unwashed masses who also had useful contributions (in addition to a lot of noise)... then things got really crowded with noise when AOL went live... And I'm sure there's a number of other BBS and/or compuserve communities I'm giving short-shrift here because I wasn't part of them in real time. The phenomenon of "September" being the time when all the new undergrads got their accounts and discovered USENET and the shenanigans that ensued was well-known. Eternal September when AOL got connected was a serious body blow. As for BBSes...I'd go so far as to say that the BBS people were the AOL people before the AOL people were the AOL people. A takeaway from both was that communities with established norms but no way beside social pressure to enforce them have a hard time scaling. USENET worked when the user population was small and mostly amenable to a set of shared goals centered around information exchange (nevermind the Jim Flemings and other well-known cranks of the world). But integrating someone into the fold took effort both on the community's part as well as the user; when it wasn't obvious that intrinsic motivation was required, or hordes of users just weren't interested, it didn't work very well. I think this is something we see over and over again with social networks. - Dan C. From mike.ab3ap at gmail.com Fri Feb 10 10:37:30 2023 From: mike.ab3ap at gmail.com (Mike Markowski) Date: Thu, 9 Feb 2023 19:37:30 -0500 Subject: [COFF] Music! Message-ID: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> This is far afield even for COFF, so apologies up front.  Machines and OSes we fondly remember get older day by day.  But many labs I worked in during undergrad & grad years and then in the workforce always had a radio going, and music never seems to age.  When I hear Earth, Wind & Fire's "September" or Doobie Brothers' "What a Fool Believes," it's RSTS/E on a PDP11/70 as a teen, my first exposure to computers. Kraftwerk and Big Audio Dynamite mean Unix with Mike Muuss at Ballistic Research Lab in the early 90s.  I had PX (military Post Exchange) privileges which Mike used to the fullest to buy fantastic lab speakers.  The old ENIAC room, our work space, had thick walls.  :-) I wonder if particular music transports any others back to computing days of old.  The current lab I'm in receives exactly 1 radio station from a local high school and streaming is blocked. Not sure that any new musical memories will be formed for my ever nearer days of retirement! Musically yours, Mike Markowski -------------- next part -------------- An HTML attachment was scrubbed... URL: From coff at tuhs.org Fri Feb 10 11:19:59 2023 From: coff at tuhs.org (segaloco via COFF) Date: Fri, 10 Feb 2023 01:19:59 +0000 Subject: [COFF] Music! In-Reply-To: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> Message-ID: This brings to mind an amusing anecdote from my lab days. Only really involves our usual fare in that I was on the other side of a wall from an RS/6000. In any case, at the time I was in metals samples digest, which basically meant all day I was pouring soils and liquids in and out of tubes, cooking em, wash rinse repeat, so typically would have the stereo blaring all day. Well an audit came through one day. I didn't know personally so was just doing the usual. Suddenly this guy in the cleanest lab coat I've ever seen in environmental walks in with a clipboard and I immediately run over and shush the radio, expecting admonishment from the managerial type escorting him about. Instead, he asks that I turn it back on as anything else would be altering my typical work day and would misrepresent my practices in the context of the audit. I'll forever remember that day as the day my tunes of the day factored into a formal audit, and not even in the disciplinary way I would've thought. Luckily after that humanizing moment, the audit itself was a breeze, I think he even glossed over a few things because he was enjoying the tunes too. So the lesson I learned that day is that everything, including your background entertainment, can be considered a condition of your working space and eliminating it for audits might actually misrepresent the nature of your work environment. - Matt G. ------- Original Message ------- On Thursday, February 9th, 2023 at 4:37 PM, Mike Markowski wrote: > This is far afield even for COFF, so apologies up front. Machines and OSes we fondly remember get older day by day. But many labs I worked in during undergrad & grad years and then in the workforce always had a radio going, and music never seems to age. When I hear Earth, Wind & Fire's "September" or Doobie Brothers' "What a Fool Believes," it's RSTS/E on a PDP11/70 as a teen, my first exposure to computers. Kraftwerk and Big Audio Dynamite mean Unix with Mike Muuss at Ballistic Research Lab in the early 90s. I had PX (military Post Exchange) privileges which Mike used to the fullest to buy fantastic lab speakers. The old ENIAC room, our work space, had thick walls. :-) > > I wonder if particular music transports any others back to computing days of old. The current lab I'm in receives exactly 1 radio station from a local high school and streaming is blocked. Not sure that any new musical memories will be formed for my ever nearer days of retirement! > > Musically yours, > Mike Markowski -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Fri Feb 10 11:43:29 2023 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 10 Feb 2023 12:43:29 +1100 (EST) Subject: [COFF] Music! In-Reply-To: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> Message-ID: On Thu, 9 Feb 2023, Mike Markowski wrote: [...] > I wonder if particular music transports any others back to computing > days of old.  The current lab I'm in receives exactly 1 radio station > from a local high school and streaming is blocked.  Not sure that any > new musical memories will be formed for my ever nearer days of > retirement! "Computer Games" by Mi-Sex (sp?), filmed at CDC HQ in North Sydney, Australia; all those spinning tapes in the background... And yep, Kraftwerk :-) I saw them in concert. -- Dave From lars at nocrew.org Fri Feb 10 16:19:30 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 06:19:30 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: (Theodore Ts'o's message of "Wed, 8 Feb 2023 21:51:15 -0500") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> Message-ID: <7wttzu3uvh.fsf@junk.nocrew.org> s/TUHS/COFF/ Theodore Ts'o wrote: > The only I saw were PC/AT's (that is, the ones with the '286 CPU) that > ran DOS and which were essentially used only to telnet to the Vax > 750's (or supdup to the MIT AI / LCS lab machines, but most > undergraduates didn't have access to those computers Any chance this DOS supdup software is still around? Was it part of PC/TCP? I searched around and found this: https://windowsbulletin.com/files/exe/ftp-software-inc/pc-tcp From lars at nocrew.org Fri Feb 10 16:35:59 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 06:35:59 +0000 Subject: [COFF] Music! In-Reply-To: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> (Mike Markowski's message of "Thu, 9 Feb 2023 19:37:30 -0500") References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> Message-ID: <7wpmai3u40.fsf@junk.nocrew.org> Mike Markowski wrote: > I wonder if particular music transports any others back to computing > days of old. The MIT AI lab seemed to favor Beatles and Bach. From lars at nocrew.org Fri Feb 10 20:12:38 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 10:12:38 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: (Jonathan Gray's message of "Fri, 10 Feb 2023 18:43:54 +1100") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> Message-ID: <7wlel54ynd.fsf@junk.nocrew.org> Jonathan Gray wrote: >> Any chance this DOS supdup software is still around? > > https://web.mit.edu/Saltzer/www/publications/pcip-1985.pdf > http://www.bitsavers.org/bits/MIT/pc-ip/ Great, thanks! It's a bit sad to read in supdup.mss "Unfortunately, very few machines have TCP/Supdup servers. The only servers known to us are on Mit-MC and Su-AI, and 4.2 Unix machines running a server we distribute." At this point, three old ITS machines had recently fallen over, one after the other, and MC was the only one left standing. But not long after, four new ones would appear. One of which is still up and running! From paul.winalski at gmail.com Sat Feb 11 01:23:25 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Fri, 10 Feb 2023 10:23:25 -0500 Subject: [COFF] Music! In-Reply-To: <7wpmai3u40.fsf@junk.nocrew.org> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <7wpmai3u40.fsf@junk.nocrew.org> Message-ID: On 2/10/23, Lars Brinkhoff wrote: > > The MIT AI lab seemed to favor Beatles and Bach. That they did. In the late '70s I was interning at IBM's Cambridge Scientific Center, which at that time was on the 4th floor of 545 Technology Square in Cambridge MA. We had about half the floor. The other half was part of MIT's AI lab. Our timesharing terminals were in a room with a rather thin wall with the AI lab on the other side. Someone in that room was debugging a music-playing program. The test song was the Beatles' "Being for the Benefit of Mr. Kite". I got to listen to that being played incessantly (and often wrongly) for an entire afternoon, sometimes with one note being held for many seconds when a breakpoint was hit. It drove me crazy. -Paul W. From imp at bsdimp.com Sat Feb 11 01:25:44 2023 From: imp at bsdimp.com (Warner Losh) Date: Fri, 10 Feb 2023 08:25:44 -0700 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: <7wlel54ynd.fsf@junk.nocrew.org> References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023, 3:12 AM Lars Brinkhoff wrote: > Jonathan Gray wrote: > >> Any chance this DOS supdup software is still around? > > > > https://web.mit.edu/Saltzer/www/publications/pcip-1985.pdf > > http://www.bitsavers.org/bits/MIT/pc-ip/ > > Great, thanks! > > It's a bit sad to read in supdup.mss "Unfortunately, very few machines > have TCP/Supdup servers. The only servers known to us are on Mit-MC and > Su-AI, and 4.2 Unix machines running a server we distribute." At this > point, three old ITS machines had recently fallen over, one after the > other, and MC was the only one left standing. But not long after, four > new ones would appear. One of which is still up and running! > I wonder if this was the same supdup that the early BSDs used to distribute their source in the 90s .. Warner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sat Feb 11 01:33:22 2023 From: crossd at gmail.com (Dan Cross) Date: Fri, 10 Feb 2023 10:33:22 -0500 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023 at 10:26 AM Warner Losh wrote: > On Fri, Feb 10, 2023, 3:12 AM Lars Brinkhoff wrote: >> Jonathan Gray wrote: >> >> Any chance this DOS supdup software is still around? >> > >> > https://web.mit.edu/Saltzer/www/publications/pcip-1985.pdf >> > http://www.bitsavers.org/bits/MIT/pc-ip/ >> >> Great, thanks! >> >> It's a bit sad to read in supdup.mss "Unfortunately, very few machines >> have TCP/Supdup servers. The only servers known to us are on Mit-MC and >> Su-AI, and 4.2 Unix machines running a server we distribute." At this >> point, three old ITS machines had recently fallen over, one after the >> other, and MC was the only one left standing. But not long after, four >> new ones would appear. One of which is still up and running! > > I wonder if this was the same supdup that the early BSDs used to distribute their source in the 90s .. I believe they are entirely different. If I recall correctly (and it's been a while...) csup, CVSup, et al were based on `sup`, which was a file distribution tool somewhat like `rdist`, which came from CMU. csup/cvsup was optimized for moving source code deltas (a la CVS repositories) around. I recall a graphical client written in Modula-3? The SUPDUP protocol used by ITS hosts was a "Display Protocol" based on TELNET: https://www.rfc-editor.org/rfc/rfc734 I think the similarity in naming was just a coincidence. - Dan C. From clemc at ccc.com Sat Feb 11 01:44:29 2023 From: clemc at ccc.com (Clem Cole) Date: Fri, 10 Feb 2023 10:44:29 -0500 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023 at 10:34 AM Dan Cross wrote: > If I recall correctly (and it's been a while...) csup, CVSup, et al > were based on `sup`, which was a file distribution tool somewhat like > `rdist`, which came from CMU. csup/cvsup was optimized for moving > source code deltas (a la CVS repositories) around. I recall a > graphical client written in Modula-3? > Right -- ??maybe?? Bob Baron or one of the Mach guys did it - used the DEC Modula-3 IIRC. > > The SUPDUP protocol used by ITS hosts was a "Display Protocol" based > on TELNET: https://www.rfc-editor.org/rfc/rfc734 Exactly - popular with the LISP hackers. I started to implement it in the CMU Distributed Front-End years ago but never finished it - I don't know if it was ever completed. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sat Feb 11 03:19:25 2023 From: crossd at gmail.com (Dan Cross) Date: Fri, 10 Feb 2023 12:19:25 -0500 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023 at 10:44 AM Clem Cole wrote: > On Fri, Feb 10, 2023 at 10:34 AM Dan Cross wrote: >> If I recall correctly (and it's been a while...) csup, CVSup, et al >> were based on `sup`, which was a file distribution tool somewhat like >> `rdist`, which came from CMU. csup/cvsup was optimized for moving >> source code deltas (a la CVS repositories) around. I recall a >> graphical client written in Modula-3? > > Right -- ??maybe?? Bob Baron or one of the Mach guys did it - used the DEC Modula-3 IIRC. That sounds right for `sup`. I vaguely remember it being associated with Coda, as well? I believe the BSDs used it to distribute source code (in addition to or instead of anoncvs?) in the 90s. It looks like CVSup was done by John Polstra in Modula-3. It was being used for FreeBSD in at least 1996, but I don't know when they started using it. It's mostly disappeared, but archive.org has a snapshot of its old web site: https://web.archive.org/web/20060103034312/http://www.cvsup.org/faq.html `csup` was a reimplementation in C (without the GUI part, I imagine) because the Modula-3 dependency was a pain. >> The SUPDUP protocol used by ITS hosts was a "Display Protocol" based >> on TELNET: https://www.rfc-editor.org/rfc/rfc734 > > Exactly - popular with the LISP hackers. I started to implement it in the CMU Distributed Front-End years ago but never finished it - I don't know if it was ever completed. Oh yeah, I imagine it was implemented on Lisp machines, probably for connecting to ITS. Lars, do you know? - Dan C. From lars at nocrew.org Sat Feb 11 04:21:13 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 18:21:13 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: (Dan Cross's message of "Fri, 10 Feb 2023 10:33:22 -0500") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: <7wh6vt4c12.fsf@junk.nocrew.org> Dan Cross wrote: > Warner Losh wrote: >>> Jonathan Gray wrote: >>> >> Any chance this DOS supdup software is still around? >>> > http://www.bitsavers.org/bits/MIT/pc-ip/ >> I wonder if this was the same supdup that the early BSDs used to >> distribute their source in the 90s .. If there was any RFC734 supdup in BSD, then I'm guessing that would have been a different one written by David Bridgham. The PC/IP one seems to be targeted specifically at MS-DOS, although cross compiled from BSD. Maybe Bridgham was involved in that one too, though. I have asked, and will report back. > I believe they are entirely different. If I recall correctly (and > it's been a while...) csup, CVSup, et al were based on `sup` If Warner was referring to those, then you would be right. From lars at nocrew.org Sat Feb 11 04:33:52 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 18:33:52 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: (Dan Cross's message of "Fri, 10 Feb 2023 12:19:25 -0500") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: <7w8rh54bfz.fsf@junk.nocrew.org> Dan Cross wrote: > Oh yeah, I imagine [SUPDUP] was implemented on Lisp machines, probably > for connecting to ITS. Lars, do you know? I don't know if it was first implemented on ITS or Lisp machines, but one of those two. SUPDUP is basically just ITS' internal terminal buffer codes for text and cursor movement, adopted as a network protocol. So certainly the first SUPDUP server would have been written for ITS. Stanford's AI lab also adopted the protocol. But outside those two sites, almost no one. The name comes from "super-duper image mode", shortened to fit ITS' file name limit of six characters. On ITS, a terminal may be opened in "ASCII mode" which means control chacters are interpreted. In "image mode" most control characters are passed through verbatim. There's a special "super-image mode" where *all* characters are passed through. So "super-duper image mode", then, alludes to the fact that even the normally invisible internal codes are passed through. From imp at bsdimp.com Sat Feb 11 04:34:33 2023 From: imp at bsdimp.com (Warner Losh) Date: Fri, 10 Feb 2023 11:34:33 -0700 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023 at 8:33 AM Dan Cross wrote: > On Fri, Feb 10, 2023 at 10:26 AM Warner Losh wrote: > > On Fri, Feb 10, 2023, 3:12 AM Lars Brinkhoff wrote: > >> Jonathan Gray wrote: > >> >> Any chance this DOS supdup software is still around? > >> > > >> > https://web.mit.edu/Saltzer/www/publications/pcip-1985.pdf > >> > http://www.bitsavers.org/bits/MIT/pc-ip/ > >> > >> Great, thanks! > >> > >> It's a bit sad to read in supdup.mss "Unfortunately, very few machines > >> have TCP/Supdup servers. The only servers known to us are on Mit-MC and > >> Su-AI, and 4.2 Unix machines running a server we distribute." At this > >> point, three old ITS machines had recently fallen over, one after the > >> other, and MC was the only one left standing. But not long after, four > >> new ones would appear. One of which is still up and running! > > > > I wonder if this was the same supdup that the early BSDs used to > distribute their source in the 90s .. > > I believe they are entirely different. > > If I recall correctly (and it's been a while...) csup, CVSup, et al > were based on `sup`, which was a file distribution tool somewhat like > `rdist`, which came from CMU. csup/cvsup was optimized for moving > source code deltas (a la CVS repositories) around. I recall a > graphical client written in Modula-3? > cvsup was written by John Polstra in Modula-3. IT had an optional GUI client. It was notable at the time for using the radical new concept of threading to stream the exchange elements of the protocol, speeding it up radically over the old sup protocol, which was entirely synchronous. In addition, it knew about the structure of RCS files, and exchanged only delta information so could make a number of interesting optimizations in transmission of data and delta data (since RCS had rules for the version number's forms, etc). It was from the very early days of the FreeBSD project (somewhere between FreeBSD 2.0 and 2.1) because the sup protocol was killing the wallnut creek CD-ROM servers with its load (a primary motivation for teaching it about the structure of RCS files). csup was a later rewrite in 'c' as the module-3 compiler fell into disrepair. It was written by Maxime Henrion and is protocol compatible with the cvsup programs. It lacked a gui, but was way more portable and didn't require that a new architecture in FreeBSD first port the Module-3 compiler to it so source updates were possible (only slight exaggeration). cvsup was ported between three or four different Modula-3 compilers as support for the language was fading away.... Both became obsolete when the FreeBSD project moved to svn. It offered not-quite-as-good, but good-enough delta updates, so cvsup and csup became relegated to the dustbin of history. > The SUPDUP protocol used by ITS hosts was a "Display Protocol" based > on TELNET: https://www.rfc-editor.org/rfc/rfc734 > > I think the similarity in naming was just a coincidence. > Now that you say that, I recall that as well... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Sat Feb 11 04:43:27 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 18:43:27 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: <7w8rh54bfz.fsf@junk.nocrew.org> (Lars Brinkhoff's message of "Fri, 10 Feb 2023 18:33:52 +0000") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> <7w8rh54bfz.fsf@junk.nocrew.org> Message-ID: <7w4jrt4b00.fsf@junk.nocrew.org> > Dan Cross wrote: >> Oh yeah, I imagine [SUPDUP] was implemented on Lisp machines, probably >> for connecting to ITS. Lars, do you know? > > I don't know if it was first implemented on ITS or Lisp machines, but > one of those two. I think this quote from the ITS program settles it. At this point the CONS prototype Lispm was only just up and running. DAM is David Moon. TITLE SUPER DUPER IMAGE TELNET ; This program allows one to log in to one ITS from another. ; The full characteristics of the terminal are preserved. ; DAM, May 1975 From imp at bsdimp.com Sat Feb 11 04:44:32 2023 From: imp at bsdimp.com (Warner Losh) Date: Fri, 10 Feb 2023 11:44:32 -0700 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: On Fri, Feb 10, 2023 at 10:20 AM Dan Cross wrote: > On Fri, Feb 10, 2023 at 10:44 AM Clem Cole wrote: > > On Fri, Feb 10, 2023 at 10:34 AM Dan Cross wrote: > >> If I recall correctly (and it's been a while...) csup, CVSup, et al > >> were based on `sup`, which was a file distribution tool somewhat like > >> `rdist`, which came from CMU. csup/cvsup was optimized for moving > >> source code deltas (a la CVS repositories) around. I recall a > >> graphical client written in Modula-3? > > > > Right -- ??maybe?? Bob Baron or one of the Mach guys did it - used the > DEC Modula-3 IIRC. > > That sounds right for `sup`. I vaguely remember it being associated > with Coda, as well? I believe the BSDs used it to distribute source > code (in addition to or instead of anoncvs?) in the 90s. > The original sup was on the 4.4BSD tapes, and was written in C' It may have come from Mach. > It looks like CVSup was done by John Polstra in Modula-3. It was being > used for FreeBSD in at least 1996, but I don't know when they started > using it. It's mostly disappeared, but archive.org has a snapshot of > its old web site: > https://web.archive.org/web/20060103034312/http://www.cvsup.org/faq.html > Yes. FreeBSD didn't offer anoncvs because CVS was so terrible at handling the high rate of change for the FreeBSD project. I don't think any of the other BSD projects ever used it officially, but I have a vague memory of there being unofficial NetBSD and OpenBSD mirrors. The latter two projects focused on using anoncvs, to the present day (though both now have git and other mirrors). DragonFlyBSD used it briefly since it started with CVS and then transitioned to git, when it stopped using it entirely. > `csup` was a reimplementation in C (without the GUI part, I imagine) > because the Modula-3 dependency was a pain. > Yes. The compilers were slow to update, the old ones stopped working due to the compilers heavy knowledge of threads, which were undergoing flux in the base FreeBSD system, so updates were always tricky. Plus there were issues with different upstreams accepting FreeBSD patches for various reasons that I have only a vague recollection of. There were two or three compilers, IIRC: The original DEC one, then an egcs based one and then a gcc one when egcs and gcc "reunited"... This was an unbelievable pain, with few people knowing how to properly do a bringup and John retiring or becoming semi-retired in this time period. Warner > >> The SUPDUP protocol used by ITS hosts was a "Display Protocol" based > >> on TELNET: https://www.rfc-editor.org/rfc/rfc734 > > > > Exactly - popular with the LISP hackers. I started to implement it in > the CMU Distributed Front-End years ago but never finished it - I don't > know if it was ever completed. > > Oh yeah, I imagine it was implemented on Lisp machines, probably for > connecting to ITS. Lars, do you know? > > - Dan C. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Sat Feb 11 04:53:32 2023 From: imp at bsdimp.com (Warner Losh) Date: Fri, 10 Feb 2023 11:53:32 -0700 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> Message-ID: [[ sorry for replying to myself ]] One or two tweaks... On Fri, Feb 10, 2023 at 11:44 AM Warner Losh wrote: > > > On Fri, Feb 10, 2023 at 10:20 AM Dan Cross wrote: > >> On Fri, Feb 10, 2023 at 10:44 AM Clem Cole wrote: >> > On Fri, Feb 10, 2023 at 10:34 AM Dan Cross wrote: >> >> If I recall correctly (and it's been a while...) csup, CVSup, et al >> >> were based on `sup`, which was a file distribution tool somewhat like >> >> `rdist`, which came from CMU. csup/cvsup was optimized for moving >> >> source code deltas (a la CVS repositories) around. I recall a >> >> graphical client written in Modula-3? >> > >> > Right -- ??maybe?? Bob Baron or one of the Mach guys did it - used the >> DEC Modula-3 IIRC. >> >> That sounds right for `sup`. I vaguely remember it being associated >> with Coda, as well? I believe the BSDs used it to distribute source >> code (in addition to or instead of anoncvs?) in the 90s. >> > > The original sup was on the 4.4BSD tapes, and was written in C' It may > have come from Mach. > > >> It looks like CVSup was done by John Polstra in Modula-3. It was being >> used for FreeBSD in at least 1996, but I don't know when they started >> using it. It's mostly disappeared, but archive.org has a snapshot of >> its old web site: >> https://web.archive.org/web/20060103034312/http://www.cvsup.org/faq.html >> > > Yes. FreeBSD didn't offer anoncvs because CVS was so terrible at handling > the > high rate of change for the FreeBSD project. I don't think any of the other > BSD projects ever used it officially, but I have a vague memory of there > being > unofficial NetBSD and OpenBSD mirrors. The latter two projects focused on > using anoncvs, to the present day (though both now have git and other > mirrors). > DragonFlyBSD used it briefly since it started with CVS and then > transitioned to > git, when it stopped using it entirely. > anoncvs didn't cache the state, but cvsupd did, so with the high number of clients FreeBSD had, cvsupd could handle the load, while anoncvs made the machine fall over a lot due to all the file I/O and some, long since fixed, bugs in the new-at-the-time unified buffer cache code. The data for cvsupd's working set could fit into memory, but cvs's scanning of the whole tree blew any caching in the kernel out of the water in addition to hitting the above bugs.... > `csup` was a reimplementation in C (without the GUI part, I imagine) >> because the Modula-3 dependency was a pain. >> > > Yes. The compilers were slow to update, the old ones stopped working > due to the compilers heavy knowledge of threads, which were undergoing > flux in the base FreeBSD system, so updates were always tricky. Plus > there were issues with different upstreams accepting FreeBSD patches > for various reasons that I have only a vague recollection of. There were > two > or three compilers, IIRC: The original DEC one, then an egcs based one > and then a gcc one when egcs and gcc "reunited"... This was an unbelievable > pain, with few people knowing how to properly do a bringup and John > retiring > or becoming semi-retired in this time period. > cvsup was there for FreeBSD/alpha, but never made the transition to powerpc, sparc64 or mips because the modula-3 compiler support for those platforms at the time was terrible to non-existent... And IIRC the transition from the DEC compiler to egcs was motivated by trying to fix this problem, but I don't think it ever really worked out that way... I also think amd64 was an issue too, but my recollections on that are fuzzy. Warner > Warner > > >> >> The SUPDUP protocol used by ITS hosts was a "Display Protocol" based >> >> on TELNET: https://www.rfc-editor.org/rfc/rfc734 >> > >> > Exactly - popular with the LISP hackers. I started to implement it in >> the CMU Distributed Front-End years ago but never finished it - I don't >> know if it was ever completed. >> >> Oh yeah, I imagine it was implemented on Lisp machines, probably for >> connecting to ITS. Lars, do you know? >> >> - Dan C. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sat Feb 11 05:05:36 2023 From: bakul at iitbombay.org (Bakul Shah) Date: Fri, 10 Feb 2023 11:05:36 -0800 Subject: [COFF] Music! In-Reply-To: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> Message-ID: <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> When I started at Fortune Systems in 1981, they had just rented a large empty warehouse space in San Carlos that was just perfect for playing music loudly! We worked crazy hours so I took in my stereo system and some records. Once the 9 to 5 people went home we cranked up the volume and listened to local college radio stations or records. Soon people started bringing in their own favorite records. Employee ages ranged from 18 to 50+ & from variety of backgrounds so we had quite an eclectic collection of music. I also recall listening to hours of various versions of "Pipeline" and later "Louie Louie" specials on KFJC! Once we moved into a new facility the dynamics and the acoustics changed so I brought the stereo system home, along with a few records that no one claimed! Somewhat related: there was enough RF leakage when the Fortune motherboard was not in the case. I could "tune into it" near a Jazz FM station. That was quite useful because the noise pattern changed depending on what the system was doing. I could be doing something else and I could hear if the system crashed or the pattern changed to something unusual! I'd probably recognize those noise patterns even now - just as most of us oldtimers recognize dialup sounds! > On Feb 9, 2023, at 5:10 PM, Mike Markowski wrote: > >  This is far afield even for COFF, so apologies up front. Machines and OSes we fondly remember get older day by day. But many labs I worked in during undergrad & grad years and then in the workforce always had a radio going, and music never seems to age. When I hear Earth, Wind & Fire's "September" or Doobie Brothers' "What a Fool Believes," it's RSTS/E on a PDP11/70 as a teen, my first exposure to computers. Kraftwerk and Big Audio Dynamite mean Unix with Mike Muuss at Ballistic Research Lab in the early 90s. I had PX (military Post Exchange) privileges which Mike used to the fullest to buy fantastic lab speakers. The old ENIAC room, our work space, had thick walls. :-) > > I wonder if particular music transports any others back to computing days of old. The current lab I'm in receives exactly 1 radio station from a local high school and streaming is blocked. Not sure that any new musical memories will be formed for my ever nearer days of retirement! > > Musically yours, > Mike Markowski -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Sat Feb 11 05:13:15 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 19:13:15 +0000 Subject: [COFF] Music! In-Reply-To: (Paul Winalski's message of "Fri, 10 Feb 2023 10:23:25 -0500") References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <7wpmai3u40.fsf@junk.nocrew.org> Message-ID: <7wzg9l2v1w.fsf@junk.nocrew.org> Paul Winalski wrote: >> The MIT AI lab seemed to favor Beatles and Bach. > That they did. In the late '70s I was interning at IBM's Cambridge > Scientific Center, which at that time was on the 4th floor of 545 > Technology Square in Cambridge MA. We had about half the floor. The > other half was part of MIT's AI lab. Our timesharing terminals were > in a room with a rather thin wall with the AI lab on the other side. > Someone in that room was debugging a music-playing program. The test > song was the Beatles' "Being for the Benefit of Mr. Kite". Thanks for the story! In the early 70s the music was mostly playing from the old PDP-6 (the loudspeakers are visible in photos from inside the lab). However, it was always located on the 9th floor. I wonder if what you heard could have been from the Logo group, which was part of the AI lab? I understand they had a "music box" for children to play with. From lars at nocrew.org Sat Feb 11 06:21:47 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 10 Feb 2023 20:21:47 +0000 Subject: [COFF] [TUHS] Re: project athena (was Re: Setting up an X Development Environment for Mac OS) In-Reply-To: <7wh6vt4c12.fsf@junk.nocrew.org> (Lars Brinkhoff's message of "Fri, 10 Feb 2023 18:21:13 +0000") References: <3e272d72-b77a-d347-b5c3-7ed19482e5af@gmail.com> <3h5FEAegoTs6FrhHODiW-rBdB59dt_Rmr4G0PIw7flqaJLsmorgPsilm4f2aJkDud-qEljDjnCJcE1uY05Iw4HNQcyNG4W3wzVlLD0UZfLg=@protonmail.com> <7wttzu3uvh.fsf@junk.nocrew.org> <7wlel54ynd.fsf@junk.nocrew.org> <7wh6vt4c12.fsf@junk.nocrew.org> Message-ID: <7wv8k92rvo.fsf@junk.nocrew.org> > If there was any RFC734 supdup in BSD, then I'm guessing that would have > been a different one written by David Bridgham. The PC/IP one seems to > be targeted specifically at MS-DOS, although cross compiled from BSD. > Maybe Bridgham was involved in that one too, though. I have asked, and > will report back. Bridgham answered: "I wrote both of those clients as well as the BSD server. That's long enough ago that I don't recall if I shared any code between them. I don't think did, other than the fact that I wrote both so they probably share similarities just because of that." From paul.winalski at gmail.com Sat Feb 11 07:09:15 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Fri, 10 Feb 2023 16:09:15 -0500 Subject: [COFF] Music! In-Reply-To: <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> Message-ID: On 2/10/23, Bakul Shah wrote: > > Somewhat related: there was enough RF leakage when the Fortune motherboard > was not in the case. I could "tune into it" near a Jazz FM station. Stan Rabinowitz, longtime member of DEC's PDP-8 software group, discovered that the same could be done with the PDP-8. He wrote a program to play music on the PDP-8 that way. > That was > quite useful because the noise pattern changed depending on what the system > was doing. Back when I was in grad school my hearing was keen enough that I could hear the high-pitch noise of television set vertical retrace and acoustic burglar alarm systems. The power supply on the VAX-11/780 emitted a steady sound at about 15KHz, which I could hear. The pitch varied depending on what the system was doing. There was a particular sequence of pitch changes when the spooling system was about to start a print job. I used to astound people by pointing at the line printer a second or two before the print job started. -Paul W. From sjenkin at canb.auug.org.au Sat Feb 11 12:53:06 2023 From: sjenkin at canb.auug.org.au (steve jenkin) Date: Sat, 11 Feb 2023 13:53:06 +1100 Subject: [COFF] Music! In-Reply-To: <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> Message-ID: In 1996, a Unix conference in Melbourne had a visit to the reconstructed first (valve) computer in Australia, CSIRAC. [ last extracts all about There was a reconstructed audio of ’tunes’ played in 1951 at a conference. I recall at the time being told that an AM radio had, at one time, been used to play tunes from the beast. They controlled square waves produced by ALU for different amplitudes on a single frequency to approximate AM. Can’t find a reference to that now :( Dave Horsfall, IIRC, was at that event & knows radio. Might correct my recollection. There was a loudspeaker connected to the machine, which seems to be all that’s written about. CSIRAC is the only pre-1950 computer in the world to be preserved. All others were broken up for scrap - their useful life over. IIRC, the console of the Manchester Mk 1 was found near a railway line, used as part of a retaining wall. It was preserved accidentally, not for any good reason but the lack of industry / imagination of Australia bureaucrats and cheap wharehouse storage :) When I worked on early IBM 370’s, older operators talked about tuning in (AM) radios to systems to monitor operation - no refs, sorry. Exactly what’s mentioned in this note and another with the line spooler. > On 11 Feb 2023, at 06:05, Bakul Shah wrote: > > Somewhat related: there was enough RF leakage when the Fortune motherboard was not in the case. I could "tune into it" near a Jazz FM station. > That was quite useful because the noise pattern changed depending on what the system was doing. ======= Long article by working expert, well illustrated. CSIRAC Since I deal with the history of computer music as I could not dedicate an article to this subject also? For many readers far from new but definitely important to form a complete picture of what happened at the turn of the fifties. In the last months of 2014, I worked to put together the facts of some pioneering experiences of computer music that took place in the United States, more or less in the same years of CSIRAC. From this point of view, the Australian experiment completes the path that I presented in October 2014 at the Conservatorio di Santa Cecilia in Rome on the occasion of the 20th Colloquium on Music Informatics. ======= CSIRAC links The Computer CSIRAC, Film, 1965. CSIRO Film Unit [ on occasion of decommissioning ] 15 min ======= The music of CSIRAC This is a very brief extract from the book Australia’s First Computer Music, available in Australia through Common Ground Publishing and internationally through Amazon. Used with kind permission. Paul Doornbusch ======= The loudspeaker: The music of CSIRAC Although the musical developments with the CSIR Mk1 were accomplished in isolation and with no precedence, it was not unique at that time. The music played by the Ferranti Mark I was recorded by the BBC in September 1951. Reconstruction of the music played by CSIRAC Audio reconstruction of CSIRAC playing Colonel Bogey (MOV 76.5 KB) (c.1951) 77kB Acknowledgements, The music of CSIRAC Sources & Links ======= The music played by CSIRAC The first programmers of the CSIR Mk1 were Geoff Hill and Trevor Pearcey. Geoff Hill had perfect pitch and came from a very musical family. Hill was the first person to program the CSIR Mk1 to play a musical melody. It was played publicly for the first public exhibition of the computer on the 7th to 9th of August in 1951, at the inaugural Conference of Automatic Computing Machines in Sydney. Audio – The technical feature more interesting, given the subject of this site, was the presence of a speaker. The fact that the CSIR MK1 was equipped with a device of this kind was not a big news, since also others computer in circulation in those years had a pretty common thing. ======= The Last of the First, CSIRAC: Australia’s First Computer (PDF 4.9 MB) Book many images, many interviews. ======= Geoffrey W Hill The CSIR Mk1 ran its first test programs in late 1949, and it was the fifth electronic stored program computer ever developed. It embodied many features novel at the time and was able to operate more than 1000 times faster than the best mechanical calculators. The machine was officially opened in 1951 and used to solve problems both for the Radiophysics Laboratory and outside organisations. It was decommissioned in 1955 and shipped to Melbourne. Geoff Hill was the main programmer at that time and he used the machine to play musical melodies. These melodies, mostly from popular songs, were; ‘Colonel Bogey’, ‘Bonnie Banks’, ‘Girl with Flaxen Hair’ and so on. The CSIR Mk1 was dismantled in mid 1955 and moved to The University of Melbourne. On 14 June 1956 the Mk1 was recommissioned and renamed CSIRAC and the new Computation Laboratory at the University of Melbourne was officially opened. CSIRAC was available as a general computing workhorse and in the period from June 1956 to June 1964 processed over 700 computing projects. In November 1964, Dr. Frank Hirst switched CSIRAC off for the last time and it was donated to the Museum of Victoria. ======= CSIRAC Collection 5 images ======= CSIRAC, The First Computer in Australia, 1949-1964 CSIRAC, designed and built by CSIR scientists, was the first stored-memory electronic computer in Australia. In 1996, CSIRAC was loaned to the Department of Computer Science at the University of Melbourne where it was reassembled and used as the centrepiece for a conference celebrating the 40th anniversary of its recommissioning in Melbourne in June 1956, which brought together computing pioneers who had worked with CSIRAC. The machine was again temporarily displayed by the Museum at its Moreland store in November 1999, at a celebration to mark its 50th anniversary. Following several years of dedicated work by Museum staff and volunteers from the CSIRAC history team at the University of Melbourne, CSIRAC was placed back on public display, forming the centrepiece of the @digital.au exhibition in the Science & Life Gallery when the new Melbourne Museum opened at Carlton Gardens in October 2000. The display presented the computer as it appeared when set up at the University of Melbourne Computation Laboratory in 1956. In November 2010, CSIRAC was transferred to a smaller display on the west side of the Lower Ground Floor Entrance Lobby at Melbourne Museum In November 2017, CSIRAC was transferred to Scienceworks in Spotswood, where it was initially held in temporary storage, before being placed on public display once again in June 2018. ======= -- Steve Jenkin, IT Systems and Design 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin From dave at horsfall.org Sat Feb 11 14:20:46 2023 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 11 Feb 2023 15:20:46 +1100 (EST) Subject: [COFF] Music! In-Reply-To: References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> Message-ID: On Sat, 11 Feb 2023, steve jenkin wrote: > In 1996, a Unix conference in Melbourne had a visit to the reconstructed > first (valve) computer in Australia, CSIRAC. [...] > Can’t find a reference to that now :( > > Dave Horsfall, IIRC, was at that event & knows radio. > Might correct my recollection. My ears are burning :-) No. I don't recall that, but I may have written about it in AUUGN (I wrote heaps of articles in those golden days). I do recall a visit to KRE's machine room, though; the PIN for his machine room was "1428" (the first four digits of the fractional part of 22/7, of course). I don't recall going to Monash, though; umm, remind me? The bloke with the TTY driver hangup? He spoke at a few AUUGs. > CSIRAC is the only pre-1950 computer in the world to be preserved. All > others were broken up for scrap - their useful life over. Sacrilege... > IIRC, the console of the Manchester Mk 1 was found near a railway line, > used as part of a retaining wall. Heh :-) I'm currently writing an emulator for that thing (just for fun, of course); now, where the hell can I find some sample programs to verify the thing? -- Dave (VK2KFU for the radio bit) From mike.ab3ap at gmail.com Sun Feb 12 02:40:02 2023 From: mike.ab3ap at gmail.com (Mike Markowski) Date: Sat, 11 Feb 2023 11:40:02 -0500 Subject: [COFF] Music! In-Reply-To: References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> Message-ID: <9ed90217-11f5-1f82-88d8-1dfe6ef15bb4@gmail.com> On 2/10/23 9:53 PM, steve jenkin wrote: > In 1996, a Unix conference in Melbourne had a visit to the reconstructed first (valve) computer in Australia, CSIRAC. > [ last extracts all about > > There was a reconstructed audio of ’tunes’ played in 1951 at a conference. > > I recall at the time being told that an AM radio had, at one time, been used to play tunes from the beast. > They controlled square waves produced by ALU for different amplitudes on a single frequency to approximate AM. This page has a bit of Beethoven's 5th using that technique on a pdp-8.  What, no stereo??  :-)   https://www.pdp8online.com/sound/sound.shtml From skogtun at gmail.com Sun Feb 12 05:24:23 2023 From: skogtun at gmail.com (Harald Arnesen) Date: Sat, 11 Feb 2023 20:24:23 +0100 Subject: [COFF] Music! In-Reply-To: References: <6f278982-4616-7fe0-ef2f-37ecd4f2ff23@gmail.com> <98FF7DD9-2CA4-4347-85B9-5D8B99395712@iitbombay.org> Message-ID: Dave Horsfall [11/02/2023 05.20]: >> IIRC, the console of the Manchester Mk 1 was found near a railway line, >> used as part of a retaining wall. > Heh :-) I'm currently writing an emulator for that thing (just for fun, of > course); now, where the hell can I find some sample programs to verify the > thing? You may check out these: -- Hilsen Harald Слава Україні! From coff at tuhs.org Thu Feb 23 05:41:35 2023 From: coff at tuhs.org (segaloco via COFF) Date: Wed, 22 Feb 2023 19:41:35 +0000 Subject: [COFF] Help needed with some old VAX Machines (moved from TUHS) Message-ID: COFF transfer, TUHS Bcc'd to know where this thread went. Between the two if you're not doing UNIX-specific things but just trying to resurrect/restore these, COFF will probably be the better place for further discussion. @OP if you're not a member of COFF already, you should be able to reach out to Warren Toomey regarding subscription. If you're feeling particularly adventurous, NetBSD still supports VAX in some manner: http://wiki.netbsd.org/ports/vax/ YMMV, but I've had some success with NetBSD on some pretty oddball stuff. As the old saying goes, "Of course it runs NetBSD". You might be able to find some old VMS stuff for them as well, but I wouldn't know where to point you other than bitsavers. There's some other archival site out there with a bunch of old DEC stuff but I can never seem to find it when I search for it, only by accident. Best of luck! - Matt G. ------- Original Message ------- On Wednesday, February 22nd, 2023 at 10:08 AM, jnc at mercury.lcs.mit.edu wrote: > > From: Maciej Jan Broniarz > > > > Our local Hackroom acquired some VAX Station machines. > > > Exactly what sort of VAXstations? There are several different kinds; one: > > http://gunkies.org/wiki/VAXstation_100 > > doesn't even include a VAX; it's just a branding deal from DEC Marketing. > Start with finding out exactly which kind(s) of VAXstation you have. > > Noel From lars at nocrew.org Thu Feb 23 16:25:56 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Thu, 23 Feb 2023 06:25:56 +0000 Subject: [COFF] [TUHS] Re: Origins of the SGS (System Generation Software) and COFF (Common Object File Format) In-Reply-To: (Clem Cole's message of "Wed, 22 Feb 2023 17:20:05 -0500") References: Message-ID: <7wh6vcx5hn.fsf@junk.nocrew.org> Clem Cole writes: > @scj - any enlightenment -- your team in USG must have been part of all > that. SMTP supports @-mention now? ;-) From crossd at gmail.com Tue Feb 28 07:04:59 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 16:04:59 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <58626A0B-EF9C-4920-8E20-CE0C4210BA6A@planet.nl> <202302272004.31RK4aGG001510@freefriends.org> <2f6faeb4-5e73-cf18-b0ff-edc3e1658f72@case.edu> <202302272022.31RKMG2L004091@freefriends.org> Message-ID: [Redirecting to COFF; TUHS to Bcc:] On Mon, Feb 27, 2023 at 3:46 PM segaloco via TUHS wrote: > I see the wisdom in your last line there, I've typed and deleted a response to this email 4 times, each one more convoluted than the last. > > The short of my stance though is, as a younger programmer (29), I am certainly not a fan of these trends that are all too common in my generation. That said, I've set foot in one single softare-related class in my life (highschool Java class) and so I don't really know what is being taught to folks going the traditional routes. All I know from my one abortive semester of college is that I didn't see a whole lot of reliance on individual exploration of concepts in classes, just everyone working to a one-size-fits-all understanding of how to be a good employee in a given subject area. Of course, this is also influenced by my philosophy and biases and such, and only represents 4-5 months of observation, but if my minimal experience with college is to be believed, I have little faith that educational programs are producing much more than meat filters between StackOverflow and . No offense to said meat filters, people gotta work, but there is something lost when the constant march of production torpedoes individual creativity. Then again, do big firms want sophisticated engineers or are we too far gone into assembly line programming with no personal connection to any of the products? I'm glad I'm as personally involved in the stuff I work with, I could see myself slipping into the same patterns of apathy if I was a nameless face in a sea of coders on some project I don't even know the legal name of any given day. This is an extraordinarily complicated subject, and it's really full of nuance. In general, I think your categorization is unfair. It sounds like you had a bad experience in your first semester of college. I can sympathize; I did too. But a thing to bear in mind is that in the first year, universities are taking kids (and yes, they are kids...sorry young folks, I don't mean that as a pejorative, but consider the context! For most young people this is their first experience living on their own, their first _real_ taste of freedom, and the first where they're about to be subject to rigorous academic expectations without a lot of systemic support) with wildly uneven academic and social backgrounds and preparing them for advanced study in a particular field...one that most haven't even identified for themselves yet. For the precocious student, this will feel stifling; for many others it will be a struggle. What, perhaps, you see as lack of intellectual curiosity may have in fact been the outward manifestations of that struggle. That said...Things are, legitimately, very different today than they were when Unix was young. The level of complexity has skyrocketed in every dimension, and things have gotten to the point where hack upon hack has congealed into a system that's nearly bursting at the seams. It's honestly amazing that anything works at all. That said, good things have been invented since 1985, and the way many of us "grew up" thinking about problems doesn't always apply anymore. The world changes; c'est la vie. - Dan C. > ------- Original Message ------- > On Monday, February 27th, 2023 at 12:22 PM, arnold at skeeve.com wrote: > > > > Chet Ramey chet.ramey at case.edu wrote: > > > > > On 2/27/23 3:04 PM, arnold at skeeve.com wrote: > > > > > > > IMHO the dependence upon IDEs is crippling; they cut & paste to the > > > > almost total exclusion of the keyboard, including when shell completion > > > > would be faster. > > > > > > Don't forget cargo-culting by pasting shell commands they got from the web > > > and barely understand, if at all. > > > > > > Yeah, really. > > > > I do what I can, but it's a very steep uphill battle, as most > > don't even understand that they're missing something, or that > > they could learn it if they wanted to. > > > > I think I'll stop ranting before I really get going. :-) > > > > Arnold From crossd at gmail.com Tue Feb 28 07:22:04 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 16:22:04 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> Message-ID: [COFF] On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: > On 2/27/23 4:01 PM, segaloco wrote: > > The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. > > Well, I suppose if it's from a trustworthy source... > > (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) I find this a little odd. If I go back to O'Reilly books from the early 90s, there was advice to do all sorts of suspect things in them, such as fetching random bits of pieces from random FTP servers (or even using email fetch tarballs [!!]). Or downloading shell archives from USENET. And of course you _can_ download the script and read through it if you want. And no one forces anyone to use `rustup`. Most vendors ship some version of Rust through their package management system these days. - Dan C. From chet.ramey at case.edu Tue Feb 28 07:42:41 2023 From: chet.ramey at case.edu (Chet Ramey) Date: Mon, 27 Feb 2023 16:42:41 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> Message-ID: <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> On 2/27/23 4:22 PM, Dan Cross wrote: > [COFF] > > On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: >> On 2/27/23 4:01 PM, segaloco wrote: >>> The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. >> >> Well, I suppose if it's from a trustworthy source... >> >> (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > > I find this a little odd. If I go back to O'Reilly books from the > early 90s, there was advice to do all sorts of suspect things in them, Sure. My sense is that the world is a less trustworthy place today, that there are more bad actors out there, and that promoting unsafe practices like this does little good. If practices like this become the norm (and they have), it gets very easy to trick someone (or worse, compromise the server and replace the script with something that does just a little bit extra). Blindly executing code you get from elsewhere as root isn't a great idea. Look at the compromises the Python community has been dealing with recently, involving replacing common packages on well-known repository sites with malicious ones. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From crossd at gmail.com Tue Feb 28 08:01:23 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 17:01:23 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> Message-ID: On Mon, Feb 27, 2023 at 4:42 PM Chet Ramey wrote: > On 2/27/23 4:22 PM, Dan Cross wrote: > > [COFF] > > > > On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: > >> On 2/27/23 4:01 PM, segaloco wrote: > >>> The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. > >> > >> Well, I suppose if it's from a trustworthy source... > >> > >> (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > > > > I find this a little odd. If I go back to O'Reilly books from the > > early 90s, there was advice to do all sorts of suspect things in them, > > Sure. My sense is that the world is a less trustworthy place today, that > there are more bad actors out there, and that promoting unsafe practices > like this does little good. If practices like this become the norm (and > they have), it gets very easy to trick someone (or worse, compromise the > server and replace the script with something that does just a little bit > extra). Blindly executing code you get from elsewhere as root isn't a > great idea. FTR, you don't usually do this as root, as by default `rustup` installs into $HOME. I'm not sure how this is any less safe than downloading, say, a tarball and running the contained `configure` script, except that in the latter case one at least has the chance to look at the script contents. > Look at the compromises the Python community has been dealing with > recently, involving replacing common packages on well-known repository > sites with malicious ones. That seems like an issue that is independent of the delivery mechanism. FWIW, when my old team brought the Rust toolchain into Google, we investigated this issue at length. Another team (Android security, I believe) had used `mrustc`, which is a Rust compiler written in C++, to bootstrap the "real" Rust compiler from source. We then downloaded and vendored each dependent crate (Rust library) that we needed, with an auditing step. So it's entirely possible to work with Rust without ever using `rustup`. - Dan C. From crossd at gmail.com Tue Feb 28 08:07:32 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 17:07:32 -0500 Subject: [COFF] [TUHS] Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: <8A7D978F-88A0-491D-90A3-A1CE843B3698@me.com> References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <8A7D978F-88A0-491D-90A3-A1CE843B3698@me.com> Message-ID: On Mon, Feb 27, 2023 at 4:52 PM Michael Stiller wrote: > > I find this a little odd. If I go back to O'Reilly books from the > > early 90s, there was advice to do all sorts of suspect things in them, > > such as fetching random bits of pieces from random FTP servers (or > > even using email fetch tarballs [!!]). Or downloading shell archives > > from USENET. > > > > And of course you _can_ download the script and read through it if you want. > > This does not help, you can detect that on the server and send something else. What? You've already downloaded the script. Once it's on your local machine, why would you download it again? > https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/ If I really wanted to see whether it had been tampered with, perhaps spin up a sacrificial machine and run, curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | tee the.script | sh and compare to the output of, curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > the.script.nopipeshell - Dan C. From crossd at gmail.com Tue Feb 28 08:17:11 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 17:17:11 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> Message-ID: On Mon, Feb 27, 2023 at 5:06 PM KenUnix wrote: > Have they not heard of common sense? Whenever I get something from git I look through it to > check for something suspicious before using it and then and only then do I do make install. Up to what size? What about the dependencies? How about the compiler that compiles it all? I have a copy of the Linux kernel I checked out on my machine; it's many millions of lines of code; sorry, I haven't read all of that. I often install things using the operating system's package manager; I haven't read through all that code, either. Life's too short as it is! > And today's cookie cutter approach to writing software means they are not learning anything > but copy paste. Where's the innovation? I imagine that when people made the switch from programming in machine code to symbolic assemblers, and then again from assembler to higher-level languages (FORTRAN! COBOL! PL/I!). And so on. Consider that, perhaps, the innovation is in how those things are all combined to do something useful for users. My ability to search, read documents, listen to music, watch real-time video, etc, is way beyond anything I could do on the machines of the early 90s. Not everything that the kids do these days is for the better, but not everything is terrible, either. This list, and TUHS, bluntly, too often makes the mistake of assuming that it is. Innovation didn't stop in 1989. - Dan C. > On Mon, Feb 27, 2023 at 4:22 PM Dan Cross wrote: >> >> [COFF] >> >> On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: >> > On 2/27/23 4:01 PM, segaloco wrote: >> > > The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. >> > >> > Well, I suppose if it's from a trustworthy source... >> > >> > (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) >> >> I find this a little odd. If I go back to O'Reilly books from the >> early 90s, there was advice to do all sorts of suspect things in them, >> such as fetching random bits of pieces from random FTP servers (or >> even using email fetch tarballs [!!]). Or downloading shell archives >> from USENET. >> >> And of course you _can_ download the script and read through it if you want. >> >> And no one forces anyone to use `rustup`. Most vendors ship some >> version of Rust through their package management system these days. >> >> - Dan C. > > > > -- > End of line > JOB TERMINATED > > From stuff at riddermarkfarm.ca Tue Feb 28 09:20:43 2023 From: stuff at riddermarkfarm.ca (Stuff Received) Date: Mon, 27 Feb 2023 18:20:43 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> Message-ID: <5b7470aa-8d77-0733-6c37-af705cad835b@riddermarkfarm.ca> On 2023-02-27 17:17, Dan Cross wrote: > On Mon, Feb 27, 2023 at 5:06 PM KenUnix wrote: >> Have they not heard of common sense? Whenever I get something from git I look through it to >> check for something suspicious before using it and then and only then do I do make install. > > Up to what size? What about the dependencies? How about the compiler > that compiles it all? > > I have a copy of the Linux kernel I checked out on my machine; it's > many millions of lines of code; sorry, I haven't read all of that. I > often install things using the operating system's package manager; I > haven't read through all that code, either. Life's too short as it is! Too true. I build in a restricted account but even so... (Decades ago, I used Harald Alvestrand's TimeTracker but disabled his UDP start-up announcer. Even though there is not much source, I never would known about it had I not read the README.) S. From chet.ramey at case.edu Tue Feb 28 09:23:32 2023 From: chet.ramey at case.edu (Chet Ramey) Date: Mon, 27 Feb 2023 18:23:32 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> Message-ID: On 2/27/23 5:01 PM, Dan Cross wrote: > On Mon, Feb 27, 2023 at 4:42 PM Chet Ramey wrote: >> On 2/27/23 4:22 PM, Dan Cross wrote: >>> [COFF] >>> >>> On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: >>>> On 2/27/23 4:01 PM, segaloco wrote: >>>>> The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. >>>> >>>> Well, I suppose if it's from a trustworthy source... >>>> >>>> (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) >>> >>> I find this a little odd. If I go back to O'Reilly books from the >>> early 90s, there was advice to do all sorts of suspect things in them, >> >> Sure. My sense is that the world is a less trustworthy place today, that >> there are more bad actors out there, and that promoting unsafe practices >> like this does little good. If practices like this become the norm (and >> they have), it gets very easy to trick someone (or worse, compromise the >> server and replace the script with something that does just a little bit >> extra). Blindly executing code you get from elsewhere as root isn't a >> great idea. > > FTR, you don't usually do this as root, as by default `rustup` > installs into $HOME. You seem to be concentrating on `rustup', which is fine, it's your preferred example. But just because you don't run `sudo sh' when using `rustup' doesn't mean there aren't a disturbingly large number of installers -- or whatever -- for which that is the recommended workflow. Nor does the fact that `rustup' is a safe example mean that this is a safe practice in general. I posit that it's a bad idea in general to blindly run scripts you download from the Internet, and it's especially bad to do it as root. Depending on how you accept risk, you can choose to do things about it, but that's often not part of recommendations. > I'm not sure how this is any less safe than downloading, say, a > tarball and running the contained `configure` script, except that in > the latter case one at least has the chance to look at the script > contents. Yeah, but with configure you don't want to. :-). In any case, if you want to, you can have a workflow where you rebuild configure yourself. > >> Look at the compromises the Python community has been dealing with >> recently, involving replacing common packages on well-known repository >> sites with malicious ones. > > That seems like an issue that is independent of the delivery mechanism. I suppose it's workflow-dependent. If your workflow for python development involves using open-source components (ctx, pytorch, etc.) you get from some repository like PyPI, you're going to be susceptible to attacks like this. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From lm at mcvoy.com Tue Feb 28 09:42:34 2023 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 27 Feb 2023 15:42:34 -0800 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> Message-ID: <20230227234234.GO12116@mcvoy.com> I think you guys are on the same team but are maybe arguing with each other more than is needed? On Mon, Feb 27, 2023 at 06:23:32PM -0500, Chet Ramey wrote: > On 2/27/23 5:01 PM, Dan Cross wrote: > >On Mon, Feb 27, 2023 at 4:42 PM Chet Ramey wrote: > >>On 2/27/23 4:22 PM, Dan Cross wrote: > >>>[COFF] > >>> > >>>On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: > >>>>On 2/27/23 4:01 PM, segaloco wrote: > >>>>>The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. > >>>> > >>>>Well, I suppose if it's from a trustworthy source... > >>>> > >>>>(Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > >>> > >>>I find this a little odd. If I go back to O'Reilly books from the > >>>early 90s, there was advice to do all sorts of suspect things in them, > >> > >>Sure. My sense is that the world is a less trustworthy place today, that > >>there are more bad actors out there, and that promoting unsafe practices > >>like this does little good. If practices like this become the norm (and > >>they have), it gets very easy to trick someone (or worse, compromise the > >>server and replace the script with something that does just a little bit > >>extra). Blindly executing code you get from elsewhere as root isn't a > >>great idea. > > > >FTR, you don't usually do this as root, as by default `rustup` > >installs into $HOME. > > You seem to be concentrating on `rustup', which is fine, it's your > preferred example. But just because you don't run `sudo sh' when using > `rustup' doesn't mean there aren't a disturbingly large number of > installers -- or whatever -- for which that is the recommended workflow. > Nor does the fact that `rustup' is a safe example mean that this is a safe > practice in general. I posit that it's a bad idea in general to blindly > run scripts you download from the Internet, and it's especially bad to > do it as root. Depending on how you accept risk, you can choose to do > things about it, but that's often not part of recommendations. > > >I'm not sure how this is any less safe than downloading, say, a > >tarball and running the contained `configure` script, except that in > >the latter case one at least has the chance to look at the script > >contents. > > Yeah, but with configure you don't want to. :-). In any case, if you want > to, you can have a workflow where you rebuild configure yourself. > > > > >>Look at the compromises the Python community has been dealing with > >>recently, involving replacing common packages on well-known repository > >>sites with malicious ones. > > > >That seems like an issue that is independent of the delivery mechanism. > > I suppose it's workflow-dependent. If your workflow for python development > involves using open-source components (ctx, pytorch, etc.) you get from > some repository like PyPI, you're going to be susceptible to attacks like > this. > > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From crossd at gmail.com Tue Feb 28 10:28:42 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 19:28:42 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> Message-ID: On Mon, Feb 27, 2023 at 6:36 PM Chet Ramey wrote: > On 2/27/23 5:01 PM, Dan Cross wrote: > > On Mon, Feb 27, 2023 at 4:42 PM Chet Ramey wrote: > >> On 2/27/23 4:22 PM, Dan Cross wrote: > >>> [COFF] > >>> > >>> On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: > >>>> On 2/27/23 4:01 PM, segaloco wrote: > >>>>> The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. > >>>> > >>>> Well, I suppose if it's from a trustworthy source... > >>>> > >>>> (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > >>> > >>> I find this a little odd. If I go back to O'Reilly books from the > >>> early 90s, there was advice to do all sorts of suspect things in them, > >> > >> Sure. My sense is that the world is a less trustworthy place today, that > >> there are more bad actors out there, and that promoting unsafe practices > >> like this does little good. If practices like this become the norm (and > >> they have), it gets very easy to trick someone (or worse, compromise the > >> server and replace the script with something that does just a little bit > >> extra). Blindly executing code you get from elsewhere as root isn't a > >> great idea. > > > > FTR, you don't usually do this as root, as by default `rustup` > > installs into $HOME. > > You seem to be concentrating on `rustup', which is fine, it's your > preferred example. Huh? Rustup is the context that this came up in: | On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: | > On 2/27/23 4:01 PM, segaloco wrote: | > The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. | | Well, I suppose if it's from a trustworthy source... | | (Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > But just because you don't run `sudo sh' when using > `rustup' doesn't mean there aren't a disturbingly large number of > installers -- or whatever -- for which that is the recommended workflow. > > Nor does the fact that `rustup' is a safe example mean that this is a safe > practice in general. I posit that it's a bad idea in general to blindly > run scripts you download from the Internet, and it's especially bad to > do it as root. Depending on how you accept risk, you can choose to do > things about it, but that's often not part of recommendations. I cannot help but point out that this is moving the goalposts somewhat from the specific context that I was responding to. If we're now talking about things in general then I agree with you. > > I'm not sure how this is any less safe than downloading, say, a > > tarball and running the contained `configure` script, except that in > > the latter case one at least has the chance to look at the script > > contents. > > Yeah, but with configure you don't want to. :-). Hah! > In any case, if you want > to, you can have a workflow where you rebuild configure yourself. This is true, but then there's the autotools source stuff that you've got to inspect as well, and on and on. Taken to its logical conclusion, we're reading the source for the package (which, if one has time, isn't necessarily a bad idea). I think in the end, running any software package involves taking a calculated risk in a number of dimensions: there's the obvious correctness and security aspects, but also legal aspects with respect to licensing and patents and so forth. For whatever it's worth, a lot of people have decided that running a script downloaded from some HTTP server somewhere is acceptable to them, provided it's decently well-known and so on. Or perhaps they just cargo-cult it and don't really think about it, which (I think) hews closer to the argument that folks here have been making. > >> Look at the compromises the Python community has been dealing with > >> recently, involving replacing common packages on well-known repository > >> sites with malicious ones. > > > > That seems like an issue that is independent of the delivery mechanism. > > I suppose it's workflow-dependent. If your workflow for python development > involves using open-source components (ctx, pytorch, etc.) you get from > some repository like PyPI, you're going to be susceptible to attacks like > this. Indeed, supply-chain attacks both for software and hardware are something that the industry generally hasn't given due consideration. I think that's (slowly) changing. Hopefully we'll see more risk analysis with respect to this going forward. Maybe the rustup folks will even change; I've put an inquiry out. - Dan C. From crossd at gmail.com Tue Feb 28 10:29:17 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 27 Feb 2023 19:29:17 -0500 Subject: [COFF] [TUHS] Re: Generational development [was Re: Re: Early GUI on Linux] In-Reply-To: <20230227234234.GO12116@mcvoy.com> References: <16241ceb-fe92-7f25-bda0-0b327847728d@case.edu> <735c811e-62ce-5384-b83f-a3887baac89d@case.edu> <5a7aa991-7656-3faf-b34a-d613736716fd@case.edu> <20230227234234.GO12116@mcvoy.com> Message-ID: On Mon, Feb 27, 2023 at 6:42 PM Larry McVoy wrote: > I think you guys are on the same team but are maybe arguing with each > other more than is needed? Hey, the fine old USENET tradition of being in a state of violent agreement! - Dan C. > On Mon, Feb 27, 2023 at 06:23:32PM -0500, Chet Ramey wrote: > > On 2/27/23 5:01 PM, Dan Cross wrote: > > >On Mon, Feb 27, 2023 at 4:42 PM Chet Ramey wrote: > > >>On 2/27/23 4:22 PM, Dan Cross wrote: > > >>>[COFF] > > >>> > > >>>On Mon, Feb 27, 2023 at 4:16 PM Chet Ramey wrote: > > >>>>On 2/27/23 4:01 PM, segaloco wrote: > > >>>>>The official Rust book lists a blind script grab from a website piped into a shell as their "official" install mechanism. > > >>>> > > >>>>Well, I suppose if it's from a trustworthy source... > > >>>> > > >>>>(Sorry, my eyes rolled so hard they're bouncing on the floor right now.) > > >>> > > >>>I find this a little odd. If I go back to O'Reilly books from the > > >>>early 90s, there was advice to do all sorts of suspect things in them, > > >> > > >>Sure. My sense is that the world is a less trustworthy place today, that > > >>there are more bad actors out there, and that promoting unsafe practices > > >>like this does little good. If practices like this become the norm (and > > >>they have), it gets very easy to trick someone (or worse, compromise the > > >>server and replace the script with something that does just a little bit > > >>extra). Blindly executing code you get from elsewhere as root isn't a > > >>great idea. > > > > > >FTR, you don't usually do this as root, as by default `rustup` > > >installs into $HOME. > > > > You seem to be concentrating on `rustup', which is fine, it's your > > preferred example. But just because you don't run `sudo sh' when using > > `rustup' doesn't mean there aren't a disturbingly large number of > > installers -- or whatever -- for which that is the recommended workflow. > > Nor does the fact that `rustup' is a safe example mean that this is a safe > > practice in general. I posit that it's a bad idea in general to blindly > > run scripts you download from the Internet, and it's especially bad to > > do it as root. Depending on how you accept risk, you can choose to do > > things about it, but that's often not part of recommendations. > > > > >I'm not sure how this is any less safe than downloading, say, a > > >tarball and running the contained `configure` script, except that in > > >the latter case one at least has the chance to look at the script > > >contents. > > > > Yeah, but with configure you don't want to. :-). In any case, if you want > > to, you can have a workflow where you rebuild configure yourself. > > > > > > > >>Look at the compromises the Python community has been dealing with > > >>recently, involving replacing common packages on well-known repository > > >>sites with malicious ones. > > > > > >That seems like an issue that is independent of the delivery mechanism. > > > > I suppose it's workflow-dependent. If your workflow for python development > > involves using open-source components (ctx, pytorch, etc.) you get from > > some repository like PyPI, you're going to be susceptible to attacks like > > this. > > > > > > -- > > ``The lyf so short, the craft so long to lerne.'' - Chaucer > > ``Ars longa, vita brevis'' - Hippocrates > > Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ > > -- > --- > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat