What is ECOFF?

[nocash]

The PSX-SDK Programmer Tool CDROM contains some COFF files, for example

Code: Select all

CLEANMEM        extended MIPS COFF (with 38h-byte optional "a.out" header)
METROWER\*.O    extended MIPS COFF (with 38h-byte optional "a.out" header, and big-endian headers)
EMU387          standard I386 COFF (with 1Ch-byte optional "a.out" header)

The "extended MIPS COFF" files seem to be known as "ECOFF", but I can't figure where that "ECOFF" name comes from. Was it used in official MIPS docs, or PSX-SDK docs... or is it just some inofficial nickname?

And is there some documentation for that ECOFF format?

Oh, and those METROWER\*.O files with big-endian headers: Is that a common thing in COFF variants to use big-endian values in the file headers? Or does that only occur in MIPS ECOFF files?

(It's already difficult enough to detect COFF files, even if they were all little-endian).

Wikipedia has an article about ECOFF, which seems to attempt to describe 32bit MIPS executables, and includes a link to an "official" document. But that document seems to describe a format called "eCOFF" for 64bit "Tru64" executables (with 50h-byte optional "a.out" header). It does very much look as if they have accidently described two (or more) entirely different COFF variants in the same article.

PS. Are there any official tools to convert MIPS ECOFF the PS-X EXE?
I know that there are homebrew tools like "eco2exe", but I am not too sure if they are properly parsing things like initial stack pointer.

[gwald]

This is too low level for me and I haven't looked at this stuff in many years, but I'll give it a crack.
ECOFF is the Net Yaroze binary executable format, I'm not sure what the differences is between COFF and ECOFF...


AI answer:

COFF and ECOFF are both file formats for executable, object code, and shared libraries used on Unix systems. The main difference between them is that ECOFF is an extension of the COFF specification, designed for the MIPS platform, and was used by DEC Ultrix and Tru64 (previously Digital Unix and OSF/1), SGI Irix, Linux/MIPS and the Net Yaroze.

COFF was introduced in Unix System V and was used on Unix systems, but it was largely replaced by ELF, introduced with SVR4. COFF and its variants, such as XCOFF and ECOFF, continue to be used on some Unix-like systems, on Microsoft Windows (Portable Executable), in UEFI environments, and in some embedded development systems.

ECOFF was developed for the MIPS platform and was used by DEC Ultrix and Tru64 (previously Digital Unix and OSF/1), SGI Irix, Linux/MIPS and the Net Yaroze. It is an extended version of the COFF specification, designed to support the specific requirements of the MIPS architecture.

In summary, the main difference between COFF and ECOFF is that ECOFF is an extension of the COFF specification, designed for the MIPS platform, while COFF is a file format used on Unix systems, which was largely replaced by ELF.

I think they come from the old 80's-90's unix systems way of doing DLL's at least that is what Net Yaroze uses ECOFF for, libps.exe is loaded in RAM when the Siocons terminal screen loads. The Net Yaroze executable programs are smaller in size because the API is already loaded in RAM.

I'm not an expert in ECOFF (or other binary executable formats) but I do maintain Yarexe, which is eco2exe/combine/exepatch/codeworrior in one.

> I know that there are homebrew tools like "eco2exe", but I am not too sure if they are properly parsing things like initial stack pointer.

I think it is:
https://github.com/gwald/Yarexe/blob/b4 ... exe.c#L681
And put into the PS-X EXE header at line 1082.


What I couldn't figure out is how to handle program exit:
It should return to caller, ie Net Yaroze Siocons terminal or Unirom screen

Here are some docs I found years ago that I found helpfull...

Excellent reference on the ECOFF format:
From DIGITAL UNIX: Assembly Language Programmer's Guide (Chapt 7 - Object)
https://www3.physnet.uni-hamburg.de/phy ... /TITLE.HTM


It's part of DIGITAL UNIX Version 4.0E or higher, November 1998 manuals (2): https://www3.physnet.uni-hamburg.de/phy ... /TITLE.HTM

It's available for download here: https://archive.org/details/digitalunix40a
Digital UNIX v4.0A (Disc 4) Documentation v4.0 (AG-QTMBA-BE)(March 1996).iso


Looks like they were going to look at ECOFF:
https://github.com/NationalSecurityAgen ... ssues/1379

[nocash]

Thanks. In general, COFF could be a raw .EXE file, or a linker .OBJ file. I don't know how the Yaroze files fit in though... if you remember how it works, maybe you could answer some questions:

Basically, you load (and execute?) the "libps.exe" file, and then load and execute the "yaroze coff" file?
Obviously they need to be loaded to different memory addresses... does the coff use a fixed address range, or do you need to deal with the coff relocation entries to map the coff to anywhere where you have free memory?
And the coff should be able to call functions in libps.exe... is there a function vector table somewhere, or does it need parsing external symbols in the coff's symbol table?

I've looked deeper into differences between COFF and ECOFF (and 64bit eCOFF).

COFF and ECOFF seem to be mostly compatble when it comes to loading and executing the binary sections, the big difference is the symbol table for linking and debugging, the differences start in the file header (where COFF's "number of symbols" is replaced by ECOFF's "size of symbol table header"), and goes on with that ECOFF's new symbol table header (which didn't exist in COFF at all).

ECOFF and 64bit ECOFF are apparently really the "same" format (they do both have that new symbol table header). But the structures differ a bit because many 32bit entries were replaced by 64bit entries, and the entries are reordered in some cases (to have the 64bit values all grouped at an 8-byte aligned location), and 64bit ECOFF has some extra/padding entries and partiallly removed cprmask entries.

Btw. what exactly are "gprmask" and "cprmask[0..3]" in ECOFF?
They are described as "general purpose register mask" and "co-processor register masks" (supposedly for cop0..cop3).
My first theory was something like control registers to configure/enable the coprocessors.
Different METRWER\LIB\*.A\*.o files have different "random" values in gprmask, apparently always with bit0=0, so my new theory is that bit1..31 are related to CPU register r1..r31 (dunno what for though, maybe indicating that the registers are (not) used or changed by the program code)?
The four cprmask values seem to be always zero (even in METROWER\LIB\LIBGTE.A\*.o). And the PSX GTE has 64 registers, so the 32bit mask cannot refer to one-flag-per-register. Hmmm, outside of the PSX world, it might be useful to have some kind of "requires FPU" flag somewhere in cprmask(?)

[gwald]

Basically, you load (and execute?) the "libps.exe" file,
libps.exe is like the DLL preloaded into RAM, IIRC it's just a normal PSX-EXE file with the library object code, I dont think its runnable tho, I haven't checked.


and then load and execute the "yaroze coff" file?
No, TMK the ECOFF is PC-DOS side only format, Siocons (the DOS application which sends the ECOFF to the Net Yaroze) reads the ECOFF and it converts it to ELF and then sends that to Net Yaroze MON.EXE PS-X EXE brick screen terminal program running on the playstation... this is the same as what eco2exe does.

Obviously they need to be loaded to different memory addresses... does the coff use a fixed address range, or do you need to deal with the coff relocation entries to map the coff to anywhere where you have free memory?

libps.exe is always loaded to the same address, the ECOFF can be linked and loaded to any address in RAM, like a normal PS-X EXE.

And the coff should be able to call functions in libps.exe...
Yes, well the ECOFF converted to PS-X ELF does.



Is there a function vector table somewhere, or does it need parsing external symbols in the coff's symbol table?

I really dont know, but I would assume the Net Yaroze DOS compiler (ECOFF version of GCC,DJGPP) has the local libps.a (40kb) library as symbol tables pointing to the libps.exe in RAM... this also includes libc functions, so the whole SDK's API is libps.exe(512kb).


Btw. what exactly are "gprmask" and "cprmask[0..3]" in ECOFF?
I dont know, I pretty much just copied what I saw in other peoples code... maybe refer to the DIGITAL UNIX docs above?

Net Yaroze Code warrior isn't ECOFF, the debug binaries .pse I couldn't figure out how to process them for yarexe, these are normally not released anyway, but the release binaries .pxe are PS-X ELF binary format, that need a little massaging.

Found out, from the NY usenet:

Subject: Re: Target management
Date: Sun, 27 Jul 97 01:12:33 GMT
From: Joel Sumner <[email protected]>
Newsgroups: scea.yaroze.programming.codewarrior


>What is going on with the .pse files. What are these puppies? The target
>file under Target/MIPS Project is specified as a .pse file, yet .pxe files
>are created as well and these are the playstation executables. Any
>enlightenment?

I think this is buried in the Targeting PlayStation manual somewhere around
the description for the MIPS Linker Panel in the IDE. The .pse file is a
combination of the executable code and the debug information (encapsulated in
ELF/DWARF format in case you were wondering). This file is used by the
debugger.

The .pxe file is in the format PSX-EXE which is a Sony format for Playstation
Executable files. This is the file which would go on a CD that the
professionals create. This is just the program code without any debug symbols.
The PlayStation Postlinker turns a .pse file into a .pxe file.

For some reason, SIOCONS doesn't support PSX-EXE format, only ECOFF (which the
GNU tools generate). Thus, you need to use PSComUtil to download the .pxe
files. If I get a spare second, I'll post the PSX-EXE file format up
somewhere. Is anyone interested in it?

-Joel

I dont know what the Pro code warrior does tho.

So, I would guess the reason Net Yaroze (GCC/DJGPP) used ECOFF is maybe because there was a ECOFF version of GDB and ELF/DWARF didn't exist in early 90's GCC/GDB or MIPS support?... I'm not too sure. Net Yaroze GDB (DOS app) uses the HSS protocol (hssutils-0.1.2), and the Net Yaroze side of remote debugger is HSS.EXE (Net Yaroze PS-X EXE app).

Sony shared the GNU code and patches for GCC/GDB and Siocons
I uploaded GCC/GDB here: https://archive.org/details/NetYaroze-GNU-SOURCES

[gwald]

I upload an old out of print 1988 COFF book here.