Another annoying bug of CHD/CHDMAN.
http://github.com/mamedev/mame/issues/10308
CHD Disk Image Format (MAME)
It doesn't store CUE Index values?
Yes, that's one of the basic flaws. The CHD format as such could store them in the subchannel data, but CHDMAN is just ignoring most of the CUE Index values.
One could convert CUE/BIN to TOC/BIN (the BIN with 990h-byte sectors) and then to CHD. If there's no tool that can do that: Convert CUE/BIN to CCD/IMG/SUB and then to TOC/BIN (maybe such tools exist) and then to CHD. That kind of toolchains are likely to imply conversion errors, but it should work in theory (except that CHD will then happily erase all Index 0 sectors).
CHD is a rather lossy compression format, and it's a bit silly to use lossless FLAC audio compression without properly preserving the more basic subchannel data. And it's totally overcomplicated with the metadata stuff with different sector sizes. It would have been easier to store all sectors always in 930h+60h byte format (and to store the TOC also in that same format instead of coming up with metadata stuff).
Anyways, I've experimented with a few more sector sizes... and checked how they are stored in the old CHCD metadata format... that should be fully solved now:
Track types:
I think I do now understand all relevant CDROM-related parts of the CHD format. Well, except two small details:
For tracks, is there a difference between "MODE2" and "MODE2_FORM_MIX"?
For subchannels, what exactly is "RW" versus "RW_RAW"?
There are two common ways for storing subchannels in 60h bytes, with differently arranged/interleaved bits (or "cooked" bits as they are called in CHD, whatever "cooking" means in that context").
In the CLAYS sample that I had uploaded some days ago, I've marked the subchannels as "RW" and stored them as 60h-byte array in this format:
Though I guess there's a 50:50 chance that it's exactly the wrong format, and maybe above should have been called "RW_RAW"? Getting that wrong would be a major mistake. It would be nice to have a test case for that... but that might be difficult...
Quick question: Do you know a way to create or test CHD files with subchannel data?
The problem is that CHDMAN doesn't check if RW or RW_RAW contain the expected data format. If you feed it with wrong data then it will just create a compressed file with that data. One could test a CHD with subchannel data with emulators that do support and use and require subchannel data (eg. for libcrypted games). And then hope that the emulator is actually distinguising RW and RW_RAW correctly as it was originally intended by the CHD developer.
To unravel what was intended:
CHD is just adopting the format/names RW and RW_RAW from CDRDAO's TOC files.
CDRDAO has apparently adopted the format/names RW and RW_RAW from SCSI, ATAPI, or ASPI drivers.
CHD defines RW and RW_RAW as so:
CDRDAO defines RW and RW_RAW as so:
That, hmmm, going by that definitions, the terms "uninterleaved" and "interleaved" are both referring to RW_RAW, that is perfectly unclear - help!
Yes, that's one of the basic flaws. The CHD format as such could store them in the subchannel data, but CHDMAN is just ignoring most of the CUE Index values.
One could convert CUE/BIN to TOC/BIN (the BIN with 990h-byte sectors) and then to CHD. If there's no tool that can do that: Convert CUE/BIN to CCD/IMG/SUB and then to TOC/BIN (maybe such tools exist) and then to CHD. That kind of toolchains are likely to imply conversion errors, but it should work in theory (except that CHD will then happily erase all Index 0 sectors).
CHD is a rather lossy compression format, and it's a bit silly to use lossless FLAC audio compression without properly preserving the more basic subchannel data. And it's totally overcomplicated with the metadata stuff with different sector sizes. It would have been easier to store all sectors always in 930h+60h byte format (and to store the TOC also in that same format instead of coming up with metadata stuff).
Anyways, I've experimented with a few more sector sizes... and checked how they are stored in the old CHCD metadata format... that should be fully solved now:
Code: Select all
CHCD Metadata (94Ch bytes, plus 10h-byte metadata header)
000h 4 Number of tracks (N) (1..99)
004h N*18h Track entries
... .. Zeropadding to 94Ch-byte size (when less than 99 tracks)
Track entries:
000h 4 Track Type (0..7, CHCD=# in table below) (eg. 6=MODE2_RAW)
004h 4 Subchannel Type (0=RW, 1=RW_RAW, 2=None)
008h 4 Sector Size (800h, 914h, 920h or 930h)
00Ch 4 Subchannel Size (0 or 60h)
010h 4 Number of Frames (aka number of sectors)
014h 4 Padding Frames (0..3) (to make Total Frames a multiple of 4)
Code: Select all
For TYPE and PGTYPE (and CHCD numeric type 0..7):
"MODE1/2048" or "MODE1" CHCD=0 800h-byte ;\Data Mode1
"MODE1/2352" or "MODE1_RAW" CHCD=1 930h-byte ;/
"MODE2/2336" or "MODE2" ;\dupe? CHCD=2 920h-byte ;\.
"MODE2/2336" or "MODE2_FORM_MIX" ;/ CHCD=5 920h-byte ;
"MODE2/2048" or "MODE2_FORM1" CHCD=3 800h-byte ; Data Mode2
"MODE2/2324" or "MODE2_FORM2" CHCD=4 914h-byte ;
"MODE2/2352" or "MODE2_RAW" or "CDI/2352" CHCD=6 930h-byte ;/
"AUDIO" (stored as big-endian samples!!!) CHCD=7 930h-byte ;-Audio CD-DA
For tracks, is there a difference between "MODE2" and "MODE2_FORM_MIX"?
For subchannels, what exactly is "RW" versus "RW_RAW"?
There are two common ways for storing subchannels in 60h bytes, with differently arranged/interleaved bits (or "cooked" bits as they are called in CHD, whatever "cooking" means in that context").
In the CLAYS sample that I had uploaded some days ago, I've marked the subchannels as "RW" and stored them as 60h-byte array in this format:
Code: Select all
00 00 00 00 00 00 00 00 00 00 00 00 ;P
01 01 01 00 00 00 00 00 02 00 5A 28 ;Q (for first sector at 00:02:00)
00 00 00 00 00 00 00 00 00 00 00 00 ;R
00 00 00 00 00 00 00 00 00 00 00 00 ;S
00 00 00 00 00 00 00 00 00 00 00 00 ;T
00 00 00 00 00 00 00 00 00 00 00 00 ;U
00 00 00 00 00 00 00 00 00 00 00 00 ;V
00 00 00 00 00 00 00 00 00 00 00 00 ;W
Quick question: Do you know a way to create or test CHD files with subchannel data?
The problem is that CHDMAN doesn't check if RW or RW_RAW contain the expected data format. If you feed it with wrong data then it will just create a compressed file with that data. One could test a CHD with subchannel data with emulators that do support and use and require subchannel data (eg. for libcrypted games). And then hope that the emulator is actually distinguising RW and RW_RAW correctly as it was originally intended by the CHD developer.
To unravel what was intended:
CHD is just adopting the format/names RW and RW_RAW from CDRDAO's TOC files.
CDRDAO has apparently adopted the format/names RW and RW_RAW from SCSI, ATAPI, or ASPI drivers.
CHD defines RW and RW_RAW as so:
Code: Select all
"RW" normal "cooked" 96 bytes per sector
"RW_RAW" raw uninterleaved 96 bytes per sector
Code: Select all
RW: packed R-W sub-channel data (96 bytes, L-EC data will be generated if required),
RW_RAW: raw R-W sub-channel data (interleaved and L-EC data already calculated, 96 bytes).
If you mean like the .SBI, I guess it's not possible. You need some external file, you cannot store it inside the .CHD.nocash wrote: Do you know a way to create or test CHD files with subchannel data?
http://forum.fobby.net/index.php?t=msg&goto=6645
http://wiki.recalbox.com/en/tutorials/u ... ion/chdmanwiki.recalbox.com wrote: For PS1 games protected by LibCrypt, normally you have SBI (Subchannel Information) files, you keep them and put them with the CHDs, otherwise your games will not pass.
I meant the whole subchannel data, for example something like CloneCD's .SUB files. Some Libcrypted games might be dumped in that format.
The .SBI format contains only a summary of the special Libcrypt related sectors. But yes, one could also create yet another conversion tool for converting .SBI to .SUB, and then do further conversions to end up with a hopefully intact CHD file.
Ideally, CHDMAN (or any CHDMAN-clone) should do all those things automatically, without needing external tools to convert CUE/BIN/SBI to CCD/IMG/SUB to TOC/BIN to CHD.
The downside is that CHD cannot filter-out standard subchannel values, so the subchannel data won't compress very well (it would become a lot bigger than the SBI file).
I am getting the feeling that CHD is only rhetorically supporting subchannels - without ever having used that feature in practice. As far as I remember, the https://github.com/mamedev/mame/issues/10308 page mentioned that even MAME cannot read subchannel data from CHD files.
The .SBI format contains only a summary of the special Libcrypt related sectors. But yes, one could also create yet another conversion tool for converting .SBI to .SUB, and then do further conversions to end up with a hopefully intact CHD file.
Ideally, CHDMAN (or any CHDMAN-clone) should do all those things automatically, without needing external tools to convert CUE/BIN/SBI to CCD/IMG/SUB to TOC/BIN to CHD.
The downside is that CHD cannot filter-out standard subchannel values, so the subchannel data won't compress very well (it would become a lot bigger than the SBI file).
I am getting the feeling that CHD is only rhetorically supporting subchannels - without ever having used that feature in practice. As far as I remember, the https://github.com/mamedev/mame/issues/10308 page mentioned that even MAME cannot read subchannel data from CHD files.
I guess the difference is the sub-header?nocash wrote: For tracks, is there a difference between "MODE2" and "MODE2_FORM_MIX"?
In the CDRDAO.
MODE2:2336 bytes
MODE2_FORM_MIX:2336 bytes including the sub-header
Sub-Headers
http://www.musoware.com/wiki/index.php? ... ub-Headers
So for now we can classify CHD as a lossy compression format until they fix the bug for the audio track indexes beyond 01 in the extracted .CUE? I don't know why they're not alarmed by this bug, looks like the team knew about it and still not fixing it since CHDv3? How come they can tolerate it if the purpose of MAME is about preserving things?
The sentence "including the sub-header" is just there for confusion (it's pretty much impossible not to include the sub-header in those sectors). The sub-header is stored twice in MODE2 sectors (as sub-header, and copy of sub-header):
http://problemkaputt.de/psxspx-cdrom-se ... coding.htm
Not that! That seems to be for album covers or streaming service or MP3 tags or whatever. CDROM sub-headers are 4-byte tall (most importantly used to distinguish between normal data sectors and XA-ADPCM audio sectors):null wrote: ↑October 27th, 2022, 5:31 pm Sub-Headers
http://www.musoware.com/wiki/index.php? ... ub-Headers
http://problemkaputt.de/psxspx-cdrom-xa ... rleave.htm
It's not only the Index 0 sectors that are lost, it's also the whole Lead-In area that's missing, and that can contain things like Disc Type, First and Last Track number, Lead-Out location, Session info, CATALOG, ISRC, CD-TEXT, those are rarely needed, but any half-decent CDROM format is able to store that information. Some formats are also storing Error flags (I've never dealt with that, but it might be important for some consoles other than PSX).
That would require a good understanding of what is on CD, how is it stored on a CD, what can be dumped, how is that info stored in CUE/CCD/NRG/TOC/MDS/etc. files. Plus, they cannot know what is currently stored in CHD files (until I've finished writing the documentation for them). And, where they are now is that they don't even know how to compile the CHDMAN source code without linking in unneccessary crap (the EXE should be 250Kbytes max and work on any computer).
CHDMAN v0.249
You do not have the required permissions to view the files attached to this post.
Looks like the beta prototypes in hidden palace uses CloneCD with SUB you can test some of those demos. You can download it directly in their site.
http://hiddenpalace.org/Category:PlayStation_prototypes
http://hiddenpalace.org/Category:PlayStation_prototypes
Some bugs about the CDI.
[CDI/2352]/redump (CD-i) #2784
http://github.com/mamedev/mame/issues/2784
[CDI/2352]/redump (CD-i) #2784
http://github.com/mamedev/mame/issues/2784
http://www.gnu.org/software/ccd2cue/man ... _0029.htmlGNU Compact Disc fields wrote: Value/Description
AUDIO - Audio/Music (2352 — 588 samples)
CDG - Karaoke CD+G (2448)
MODE1/2048 - CD-ROM Mode 1 Data (cooked)
MODE1/2352 - CD-ROM Mode 1 Data (raw)
MODE2/2048 - CD-ROM XA Mode 2 Data (form 1) *
MODE2/2324 - CD-ROM XA Mode 2 Data (form 2) *
MODE2/2336 - CD-ROM XA Mode 2 Data (form mix)
MODE2/2352 - CD-ROM XA Mode 2 Data (raw)
CDI/2336 - CDI Mode 2 Data
CDI/2352 - CDI Mode 2 Data
The modes marked with ‘*’ are not defined in the original CUE sheet format specification.
nocash wrote: For tracks, is there a difference between "MODE2" and "MODE2_FORM_MIX"?
According to claunia's source code MODE2 is FORMLESS while the MODE2_FORM_MIX is mixed forms. And there's some info in the Google Patents CDROM Decoder page tells that FORMLESS replaces the subheader with the 2336 bytes user data next to header. So technically there's no subheader for MODE2_FORMLESS, only MODE2_FORM_MIX have it.nocash wrote: The sentence "including the sub-header" is just there for confusion (it's pretty much impossible not to include the sub-header in those sectors).
http://raw.githubusercontent.com/aaru-d ... nstants.cs
http://patents.google.com/patent/KR100424230B1/en
ps. MODE2_FORMLESS is also called MODE2_FORM0
Ah, yes, that might be where the two "different" formats came from...
So the formats should be identical - and the only "difference" is whether the recurding tool has made assumptions or verifications on what is stored in the file.
Code: Select all
MODE2 "formless" --> subheader MAY contain FORM1 or FORM2 or WHATEVER info (the recording tool didn't care about that)
MODE2_FORM_MIX --> subheader SHALL contain FORM1 or FORM2 info (the recording tool did think/know/verify that)
No change, just says missing KERNEL32.DLL:AddVectoredExceptionHandler as ever since.
Does that help as test case? I mean, you will need some solid criminal energy to find or create a test case. Like, intentionally creating a CHD file with some kind of broken or nonstandard subchannel data, and then checking if any emulators do respond as expected when reading the corresponding sectors.
The easiest test cases I could think of are:
- Libcrypted PSX games.
- Audio Discs with some "visible" glitch (eg. flipping Track number 1-2-1-2-1-2-1-2-1-2-1-2 during playback).
Or, try to find people who do still own old computers with physical CDROM drive and CDRDAO software, and try to ask them if they could record a regular CDROM or Audio disc as TOC/BIN image with "--read-subchan" option (albeit risking to get told to go fuck yourself).
I can't read the bug report on the github page... and the post that you've quoted doesn't mention any bugs?null wrote: ↑November 3rd, 2022, 3:40 am Some bugs about the CDI.
[CDI/2352]/redump (CD-i) #2784
http://github.com/mamedev/mame/issues/2784http://www.gnu.org/software/ccd2cue/man ... _0029.htmlGNU Compact Disc fields wrote: Value/Description
AUDIO - Audio/Music (2352 — 588 samples)
CDG - Karaoke CD+G (2448)
MODE1/2048 - CD-ROM Mode 1 Data (cooked)
MODE1/2352 - CD-ROM Mode 1 Data (raw)
MODE2/2048 - CD-ROM XA Mode 2 Data (form 1) *
MODE2/2324 - CD-ROM XA Mode 2 Data (form 2) *
MODE2/2336 - CD-ROM XA Mode 2 Data (form mix)
MODE2/2352 - CD-ROM XA Mode 2 Data (raw)
CDI/2336 - CDI Mode 2 Data
CDI/2352 - CDI Mode 2 Data
The modes marked with ‘*’ are not defined in the original CUE sheet format specification.
The CHDMAN didn't want to accept the CDI/2352 track in a CUE. The error message is "ERROR: Unknown track type [CDI/2352]. Contact MAMEDEV". The workaround is to change the CDI/2352 to MODE2/2352 they say but it looks no one tested it yet so I can't proved if this is a working FIX solution. The issue is still open so there's no hard-coded fix yet in the latest CHDMAN. It's just a minor bug anyway but still a bug.nocash wrote: I can't read the bug report on the github page... and the post that you've quoted doesn't mention any bugs?
I`ve released no$psx v2.2 with CHD support.
And the specs for the CHD format can be found here:
http://problemkaputt.de/psxspx-cdrom-di ... d-mame.htm
And the specs for the CHD format can be found here:
http://problemkaputt.de/psxspx-cdrom-di ... d-mame.htm
About the MAME v0.255 DVD support, they classified PSP's UMD as DVD.MAME History wrote: MAME v0.255 - Add DVD support
MAME v0.262 - Added support for Zstandard compression
MAME v0.265 - Added multi-session disc support (thanks WindyFairy)
zlib/DEFLATE and Zstandard both have the same compression ratio but zstd wins in compression/decompression speed.PPSSPP wrote: PSP .ISO to .CHD
chdman createdvd -hs 2048 -i game.iso -o game.chd
PSP .ISO to .CHD only with Zstandard compression
chdman createdvd -hs 2048 -i game.iso -o game.chd -c zstd
PSP .CHD to .ISO
chdman extractdvd -hs 2048 -i game.chd -o game.iso
I don't know about multi-session discs and it looks like PSX doesn't use these. Atari Jaguar and Dreamcast uses these type of CDs.
Does using this type of .CUE directive can give multidisc PSX games multi-session capability? Like what Sony did to their .PBP PSX games?
Code: Select all
REM SESSION 010203...
You do not have the required permissions to view the files attached to this post.
The WIN32 version has found a new way to make it won't work: v0.265 now says missing KERNEL32.DLL:GetModuleHandleExW.
For Zstandard, a sample CHD file nice, like those posted above for other methods.
All those extra compression methods are getting quite annoying (but maybe I'll like it better if it should turn out to be actually faster on my old PC).
Sessions... the .PBP PSX games have multi-session?
Multi-session is interesting in general, and it could be theoretically used on PSX, but as far as I know it wan't ever used on retail PSX discs.
REM SESSION ## seems to be an inoffical CUE extension, supported only by some tools, and it seems to exist in different flavors (with/without specifying the LEAD-OUT and LEAD-IN between sessions).
Sample files in CHD and CUE/BIN format would be nice to have, best including corner cases where the ISO filesystem in session 2 is referring to some files in session 1.
(For more corner cases, sessions could theoretically mark audio tracks in previous sessions as "deleted/to be skipped"... but I guess that feature was never really used or supported on real hardware, so it's best to ignore that).
DVD / PSP UMD. Good to know. Although not needed for PSX emus, except for showing error messages... so it would be nice to have a small CHD sample file there, too.
Github is meanwhile fully blocking my browser. Except, the github "raw" pages are still visible (but it's impossible to find them without access to the search or directory pages).
I could theoretically find them using a newer browser (on a tablet) and then copying the URLs to my PC. Or if somebody wants to help me: Please search the MAME and CHDMAN source code pages for the following keywords: zstandard, zstd, session, dvd, umb (and post links to the corresponding "raw" pages):
For Zstandard, a sample CHD file nice, like those posted above for other methods.
All those extra compression methods are getting quite annoying (but maybe I'll like it better if it should turn out to be actually faster on my old PC).
Sessions... the .PBP PSX games have multi-session?
Multi-session is interesting in general, and it could be theoretically used on PSX, but as far as I know it wan't ever used on retail PSX discs.
REM SESSION ## seems to be an inoffical CUE extension, supported only by some tools, and it seems to exist in different flavors (with/without specifying the LEAD-OUT and LEAD-IN between sessions).
Sample files in CHD and CUE/BIN format would be nice to have, best including corner cases where the ISO filesystem in session 2 is referring to some files in session 1.
(For more corner cases, sessions could theoretically mark audio tracks in previous sessions as "deleted/to be skipped"... but I guess that feature was never really used or supported on real hardware, so it's best to ignore that).
DVD / PSP UMD. Good to know. Although not needed for PSX emus, except for showing error messages... so it would be nice to have a small CHD sample file there, too.
Github is meanwhile fully blocking my browser. Except, the github "raw" pages are still visible (but it's impossible to find them without access to the search or directory pages).
I could theoretically find them using a newer browser (on a tablet) and then copying the URLs to my PC. Or if somebody wants to help me: Please search the MAME and CHDMAN source code pages for the following keywords: zstandard, zstd, session, dvd, umb (and post links to the corresponding "raw" pages):
Yes, they convert some of their multi-disc PSX games into a single .PBP for PSP's POPS. Like Final Fantasy series, Metal Gear Solid, Resident Evil, etc. They upload it on their online store PSN as PSP Eboot.pbp.nocash wrote: the .PBP PSX games have multi-session?
Samples for sessions, what do you prefer? Real demo discs of Dreamcast and Atari Jaguar or just a custom audio samples with REM SESSION ##?
And then for createdvd? Do you want a real DVD image (free clips of movies?) or real PSP demo/homebrew games or a demo/homebrew PSX games turned into a DVD using chdman?
MAME Github source code.
https://raw.githubusercontent.com/mamed ... chdman.rst
https://raw.githubusercontent.com/mamed ... dcodec.cpp
https://raw.githubusercontent.com/mamed ... chdcodec.h
https://raw.githubusercontent.com/mamed ... /unzip.cpp
https://raw.githubusercontent.com/mamed ... /gdrom.cpp
https://raw.githubusercontent.com/mamed ... csi/cd.cpp
https://raw.githubusercontent.com/mamed ... /psxcd.cpp
https://raw.githubusercontent.com/mamed ... /cdrom.cpp
https://raw.githubusercontent.com/mamed ... jaguar.cpp
https://raw.githubusercontent.com/mamed ... chdman.cpp
https://raw.githubusercontent.com/mamed ... il/chd.cpp
https://raw.githubusercontent.com/mamed ... util/chd.h
https://raw.githubusercontent.com/mamed ... extlib.lua
https://raw.githubusercontent.com/mamed ... /tools.lua
https://raw.githubusercontent.com/mamed ... rc/lib.lua
https://raw.githubusercontent.com/mamed ... c/main.lua
https://raw.githubusercontent.com/mamed ... dparty.lua
https://raw.githubusercontent.com/mamed ... 8/makefile
Who is online
Users browsing this forum: No registered users and 7 guests
, "PlayStation", 
, 
, "DUALSHOCK", "Net Yaroze" and "PSone" are registered trademarks of Sony Computer Entertainment Inc. 
.