Adequacy of EPUB?

[PeterT]

Reading this reminds me of https://www.mobileread.com/forums/sho...d.php?t=232413

[eschwartz]

Originally Posted by PeterT

Reading this reminds me of https://www.mobileread.com/forums/sho...d.php?t=232413

No, @skreutzer just wanted a tool that would convert between different formats, you'll notice he didn't try suggesting we needed a new format which works everywhere.

Also, he put in the work and created a tool. I have nothing but admiration for someone who thinks there is a problem and does something about it.


This thread, on the other hand, is a bikeshed.
Suitable only for stopping by for the sheer fun of arguing.

[ProDigit]

I got in late in this conversation, but wanted to give my 2ct:
I never found the Epub format lacking.
You can easily convert from HTML0,
You can use a reader to specify a different font in case you like,
I created a bible out of it (66 books, and about 1000 chapters), with about 6000 hyperlinks (mostly to next chapter, previous chapter, index, and other chapters, as well as a hyperlink based TOC).

Titles could be centered as well...
In fact, I thought the problem was that it was harder to make titles out of center alignment.

The only device I've tested it on which had problems processing was my aged Sony PRS505 (5+ year old technology), which I guess got overloaded with all the links.

But most other devices have more memory, and faster processing, seem to be handling the data quite fine!

[Hitch]

Originally Posted by eschwartz

No, @skreutzer just wanted a tool that would convert between different formats, you'll notice he didn't try suggesting we needed a new format which works everywhere.

Also, he put in the work and created a tool. I have nothing but admiration for someone who thinks there is a problem and does something about it.

+1 on that. Didn't want his direction; but I admire his grit.

This thread, on the other hand, is a bikeshed.
Suitable only for stopping by for the sheer fun of arguing.

Indeed. Now for some more:


@Sarmat:

for future reference, yes, it's okay to quote someone, but please do it in pieces. I am unable to easily quote you in reply. It's a convenience, please.

Yes, books are simple enough matter to be described by an XML schema.

Only if you make it rigid. You'll take the art out of making books, and you WILL make it like programming.

That is just the matter of format. Generating HTML from XML source is so trivial it can be done on an E-ink Linux device easily.

Didn't you just say, not 3 posts ago, that making HTML or XHTML out of XML was also "wrong" by your lights? Or is this now you want to have the discussion both ways?

It is always 'tagged' in colophon; it would be in proper metadata if such existed.

No. It's NOT. The colophon in DocBook (see, guys? I just KNEW that this would come back to DocBook. Always does) is just another SCHEMA. That doesn't mean that the bookmakers WILL TAG the content. Don't you understand that?

eBook makers can also just plop their ePUB2 files into Calibre, and tag the holy carp out of it, using DC metadata and made-up metadata. You see them doing that? For that matter, how many ePUBs do you see that have NO metadata at ALL? Well, that's exactly what will happen with SCHEMA.

People make books with WYSIWYG editors, that provide nice toolbars and forms for filling this kind of stuff in.

Really? What WYSIWYG editor works with the Colophon Schema, exactly? Let's see...yup, there's good old Adobe FRAMEMAKER, and there's...wait for it...Oxygen. Starting at, what, $700 for commercial or business use? Sure, I think every author ought to just hurry up and buy a copy. (I knew it, I knew it, I knew it. Sho'nuff, DocBook through and through.) And then we can all--wait for it--be using XSLT, AGAIN. This always comes back to the same thing. We've all been there and done that. This is 15-y.o. stuff, that, for whatever reason, NEVER TOOK OFF. It never caught on.

Oh, wait. there was that IE 6 WYSIWYG editor. Yup. Internet Exploder SIX. That ought to tell you how far back THIS idea goes. 2002, wasn't it?

You would create a book like a BOOK, not like a webpage with options for form fields and buttons, but none for footnotes, chapters or scene breaks.

No. That's not creating a BOOK. If I were creating a book like a bloody BOOK, I'd be using fonts with correct kerning, etc. I'd be creating ART. I'd be printing it on PAPER. HTML is far closer to this is than using SCHEMA. XML is as far from art--further than HTML, by far--as you can get.

And book producers have a myriad of options of imitating the <em> with spans and CSS. With a real format, there is only one way to do that, which all readers, converters and editors support.

No,see--this is how I know you aren't really working in XML. Or SCHEMA, for that matter. If you were, you'd already KNOW that that's simply not true. You can define "names" from now until hell freezes over, and you know what? It doesn't matter. That doesn't mean, for a SECOND, that bookmaker Bob will put WHAT YOU WANT in that field. That's the part you are completely missing.

Colophon, from the last century. Unfreaking-believable.

We need a new format, that's quite clear.

You ARE talking about TEX, aren't you?

Here's the part that you don't get. For an actual, real-live commercial bookmaker--like me--your idea doesn't save me one moment of time. It doesn't give the bookmaker ANYTHING. We still have to wrap tags around everything. We'd gain nothing from it. And even if that weren't true--even if it was the next best thing, it wouldn't matter. Because--as we've all seen multiple times now--bookmakers simply make whatever is ordered by the retailers. We have NO power whatsoever to make a new format.

And given that to date, TEX is the "best" XML format in existence, you will have a big hump to get over to convince anyone to use it.

BUT, that being said, there's nothing stopping YOU from using it. Go right ahead. Buy a copy of Oxygen (or Framemaker), and build your books using XML. Slap together some XSLT, and you can build ePUBs all day long. And the ePUBs would make you happy, as the SCHEMA would still, for all intents and purposes,be included. Given that all you seem to care about is the metadata--not really the structure of the book at ALL--that should make you happy. That way, you get to have your cake and eat it too.

...and unless somebody comes along with something really bright to say, on this topic, I'm over and out.

Hitch

[Sarmat89]

Originally Posted by Hitch

making HTML or XHTML out of XML was also "wrong" by your lights?

We are talking about primitive readers which rely on HTML for displaying book pages. The quality readers can process XML directly, giving the better results due to the element standardization and rigid roles.

colophon in DocBook

No, the colophon in a physical (e-)book.

eBook makers can also just plop their ePUB2 files into Calibre, and tag the holy carp out of it, using DC metadata and made-up metadata.

They cannot do that, because the metadata scheme offered by EPUB allows you to specify such archimportant roles as 'illuminator' and 'accountant', but none that you find in any printed book.

how many ePUBs do you see that have NO metadata at ALL?

The EPUB problem is that it cannot be used for the quality work. The EPUBs being result of automatic TXT->EPUB conversion are irrelevant to discussion.

$700 for commercial or business use?

There is several FREE E-book WYSIWYG editors. Whether the underlying format is XHTML or custom XML is not important.

As for InDesign and co. EPUB export, the result should be massively tweaked to ensure the quality and compatible output. That is not different and no more difficult.

HTML is far closer to this is than using SCHEMA. XML is as far from art--further than HTML

You don't need 'art' while creating books. You need to provide semantics, and the reader software will do the rest.

No,see--this is how I know you aren't really working in XML.

Most of my E-books are in XML format. An old, incomplete format, but it doesn't mean the new, optimal one can be designed.

You ARE talking about TEX, aren't you?

TEX is not a semantic format (and hardly a format at all).

For an actual, real-live commercial bookmaker--like me--your idea doesn't save me one moment of time. It doesn't give the bookmaker ANYTHING.

The books are not created for bookmakers, they are created for distributors, who need proper metadata, and end users, who need a standardized format with high customization and predictable formatting, and obviously, proper metadata too.

Given that all you seem to care about is the metadata--not really the structure of the book at ALL

I care about the useable format, with the semantic structure included.

[Solitaire1]

I'm getting into the discussion late, but I have a few questions about the topic:

• From what I've seen here, it seems like the issue with EPUB is that it is XML based. But my understanding of XML is that it is not designed to be directly edited by humans. Instead, it is supposed to be directly edited by tools used by humans. Is that correct?
• Could many of the issues with EPUB be the result of incorrect/incomplete tagging rather than a result of the format itself? Likewise with the metadata issues?

[Sarmat89]

Originally Posted by Solitaire1

it seems like the issue with EPUB is that it is XML based.

The XML format is not an issue, the problem is that the XML container hosts HTML content.

XML is that it is not designed to be directly edited by humans. Instead, it is supposed to be directly edited by tools used by humans.

Correct. But it was purposely designed to be human-readable and handcraftable.

Could many of the issues with EPUB be the result of incorrect / incomplete tagging rather than a result of the format itself?

The real problem is that HTML cannot enforce any kind of 'correct' tagging. You may recommend people to use <p> for, you know, paragraphs, but there will always be a book that uses <div>s for that, and there is no way you can mark it as invalid.
With a specific format, it will not be possible.

[eschwartz]

But it also doesn't matter if they use a div instead of a p.

And the purpose of ebooks is not to prevent people from using it the wrong way. It is to facilitate people using it the right way.

Stupid is as stupid does, and if someone wants to lie about how the book is structured, they will find a way to do it.


The problem I keep having with your suggestions is that none of them explain how you intend to solve the problem you purport to have.

This is above and beyond the fact that EPUB is already capable of incorporating all those features anyway.

[ProDigit]

I personally dislike classes and xml.
They add a lot of garbage to a file.
Like mentioned before, if created in html0, just make sure all coding is done correctly, before adding text to the book, meaning, have the book in whatever format, and have a separate book with the layout and formatting.
Then it's as simple as merging the files.

Notepad++ for Windows, has a very easy way, through color coding, to indicate where an error may be, on a variety of code types, including html.
And through its advanced find and replace options, making chapters is a piece of cake.

I used it to get the bible framework done, with the ~5500 links. It took me only 48 hours to make, and that included learning to use notepad++.

No matter how you look at it,
There was absolutely no way to generate a file of that magnitude with class or xml, and have the ebook reader not completely go haywire.
There also was absolutely no way, to create that document any smaller in size.
Every space, code, formatting has been carefully measured,
Even with a class, I doubt the file could have been created any smaller, although once different font types come into play, it might be.
In HTML0 I refrained myself to just H1-H4 for titles (as I believe on an ebook reader h4, h5 and h6 didn't look like titles anymore compared to the book's text size).

From a coding perspective, html is the easiest to understand, has the least chance on errors, loads the fastest, and is the smallest in size, when done correctly.

[Solitaire1]

Sarmat89 wrote as part of a post:

Originally Posted by Sarmat89

The XML format is not an issue, the problem is that the XML container hosts HTML content.

I apologize for not completing my point, writing that XML might be an issue without explaining why I thought so. When I first read about XML one of the differences between HTML and XML is that HTML allows flawed tagging (such as an opening paragraph tag without the closing tag), while XML doesn't.

What stuck out to me from my reading is that an XML document must have absolutely no flaws in its tagging, and any flaws in the tagging would cause the XML document to fail. That was the reason it was supposed to be created by tools that could ensure that the resulting XML tagging is completely correct.

This could be the reason for the difficulties with EPUB ebooks. Not a flaw in the EPUB format itself but incorrect tagging. One way I think could help the situation is to have an EPUB subformat that is strictly an ebook format as an option (not supporting non-ebook features like video and audio), allowing a better focus on ebook formatting.

[Toxaris]

Originally Posted by Terisa de morgan

Tex, my God, the newest thing in the world (Not laughing at you idea but the big novelty which was proposed). And, yes, I know Tex. And yes, I've used it... without fancy editors... so using for writing a book... perhaps we can go to LaTex...

Not my idea! I know Tex since I had to use it at the university, more than 20 years ago! I do absolutly not recommend it for anything other.

Originally Posted by Hitch

No. It's NOT. The colophon in DocBook (see, guys? I just KNEW that this would come back to DocBook. Always does) is just another SCHEMA. That doesn't mean that the bookmakers WILL TAG the content. Don't you understand that?

I forgot about DOCBOOK. Yes, that seems what he is talking about. What a freaking nightmare that is.

Originally Posted by Sarmat89

We are talking about primitive readers which rely on HTML for displaying book pages. The quality readers can process XML directly, giving the better results due to the element standardization and rigid roles.

Ok, and how will the readers interpret the XML without a hard forced schema? It needs to know all the tags and the meaning of the tags. XML without a schema is meaningless. Also, if you look at novels (lets take the simplest form of books...), I can easily come up with several titles that vary in styles/layout in the text without being able to use semantics to make the distinction. If I cannot use semantics, then what? "Sorry Author X, you cannot express your creativity since it does not fit in the semantics and schema of our current format..." Yeah, right.

Originally Posted by Sarmat89

The EPUB problem is that it cannot be used for the quality work. The EPUBs being result of automatic TXT->EPUB conversion are irrelevant to discussion.
There is several FREE E-book WYSIWYG editors. Whether the underlying format is XHTML or custom XML is not important.

Ouch, have you even looked at the various books here or from others on this site? They are quality work and not automatic conversion. Automatic conversions are not quality work. They are even quality if you look at the code.

Can you tell me of two, or even one, FREE e-Book WYSIWYG editor? I know of NONE. Not real WYSIWYG. Furthermore, it is not important right? You just need to add the semantics and everything is fixed... We don't need no stinking layout apparently. Also, no self-respecting eBook maker will use a WYSIWYG editor. They will look at code, whether it is XHTML or XML doesn't matter.

Originally Posted by Sarmat89

You don't need 'art' while creating books. You need to provide semantics, and the reader software will do the rest.

Again with this nonsense. Often writers have a certain image of how the layout of the book should be, for whatever reasons. It is not up to the software. Also, without a very rigorous schema this cannot be done and that will restrict the writers/publishers too much.

Originally Posted by Sarmat89

The books are not created for bookmakers, they are created for distributors, who need proper metadata, and end users, who need a standardized format with high customization and predictable formatting, and obviously, proper metadata too.
I care about the useable format, with the semantic structure included.

Wrong, the books are created for the readers (not users, thank you very much) and the writers. They don't care for the format, but about the experience. Readers don't want to fuss with fonts, margins, indents and so on. They want to read. Some things are great if they can be changed, like text size.

I grant that ePUB has some shortcomings, like the handling of footnotes, but not nearly as big as you depict. Sure, I would like to have the Dublin Core extended with more book related metadata, but that has nothing to do with ePUB. Also, I can create my own metadata (like series information) and Calibre has certain metadata built in. Heck, there are even some readers that understand the Calibre metadata.

[Hitch]

Originally Posted by Toxaris

Not my idea! I know Tex since I had to use it at the university, more than 20 years ago! I do absolutly not recommend it for anything other.

No kidding! Now he's saying he's NOT talking about TEX. If he's not, what the HELL is he talking about? What "format" of XML eBook exists that isn't TEX?

I forgot about DOCBOOK. Yes, that seems what he is talking about. What a freaking nightmare that is.

And, now he's saying that he meant the colophon differently, so I'm completely lost. I don't understand what on earth he's talking about. He originally said, when mentioning colophon:

It is always 'tagged' in colophon; it would be in proper metadata if such existed.

to which I replied, about DocBook, and now he says:

No, the colophon in a physical (e-)book.

I'm completely lost. Neither sentence makes any sense to me. What colophon in a "physical" ebook? For that matter, what the hell is a physical eBook? I realize that the poster in question is not a native English speaker, but I don't follow this at all.

For the record: colophon is a SCHEMA that was used in DOCBOOK, for those of you late to any of the parties. DocBook, eBooks, this conversation...

Ok, and how will the readers interpret the XML without a hard forced schema? It needs to know all the tags and the meaning of the tags. XML without a schema is meaningless. Also, if you look at novels (lets take the simplest form of books...), I can easily come up with several titles that vary in styles/layout in the text without being able to use semantics to make the distinction. If I cannot use semantics, then what? "Sorry Author X, you cannot express your creativity since it does not fit in the semantics and schema of our current format..." Yeah, right.

...which is the point I'm trying to make. To me, you can only make this work if it's RIGID. And therein lies the rub.

Ouch, have you even looked at the various books here or from others on this site? They are quality work and not automatic conversion. Automatic conversions are not quality work. They are even quality if you look at the code.

Holy carp, is this guy talking about something like a basic Calibre conversion? An AbbyFineReader raw conversion? Man, no wonder he's confused. @Sarmat89: none of us are discussing "automatic" conversions, like those browser-based "convert your book to...!" websites or Smashwords. We're all discussing the ePUBs and eBooks that we all make by HAND.

Can you tell me of two, or even one, FREE e-Book WYSIWYG editor? I know of NONE. Not real WYSIWYG. Furthermore, it is not important right? You just need to add the semantics and everything is fixed... We don't need no stinking layout apparently. Also, no self-respecting eBook maker will use a WYSIWYG editor. They will look at code, whether it is XHTML or XML doesn't matter.

There are NONE for XML, that's for damn sure. I've looked trust me. The only one worth a damn is Oxygen, if you want WYSIWYG. And no: I don't know a single respectable eBookmaker that works in WYSIWYG. We all work in HTML/XHTML/CSS. In that realm, really, Sigil is the best in class, as we all know. The other WYSIWYGs are those like Jutoh, Atlantic (which is really a word-processor with an ePUB export function), etc.

Again with this nonsense. Often writers have a certain image of how the layout of the book should be, for whatever reasons. It is not up to the software. Also, without a very rigorous schema this cannot be done and that will restrict the writers/publishers too much.

OMG, they damn sure do. We're doing 3 books right now, that are ALL about the FONTS. We have another 5 that have extremely painstaking layout issues surrounding them. The rest are fairly typical, but these 8 stand out to me because they have been major pains in the tuchus. There isn't an author on earth who doesn't CARE about how their final book looks, trust me. In 3,000+ books, I've had ONE client that wanted us to upload the books for him/her, without looking at them and approving them. One! Our average book has 3-4 revision rounds, and often, at least one of them is surrounding layout. The tweaks may be minor, but bygod, they are still important to that writer or publisher.

Wrong, the books are created for the readers (not users, thank you very much) and the writers. They don't care for the format, but about the experience. Readers don't want to fuss with fonts, margins, indents and so on. They want to read. Some things are great if they can be changed, like text size.

Yes, exactly.

I grant that ePUB has some shortcomings, like the handling of footnotes, but not nearly as big as you depict. Sure, I would like to have the Dublin Core extended with more book related metadata, but that has nothing to do with ePUB. Also, I can create my own metadata (like series information) and Calibre has certain metadata built in. Heck, there are even some readers that understand the Calibre metadata.

And this seems to be the gist of the issue. I genuinely think, despite Sarmat89's assertions, that he really doesn't understand how ePUB happens. I don't say this to be derogatory; but there seems to be a major disconnect between WHAT is ePUB, itself, and WHAT is caused by how people MAKE those books. After all this, 99% of what he's complaining about still seems to be the METADATA and semantic tagging. I don't get why he thinks it's the fault of ePUB. I truly don't.

Nor, like eSchwartz, do I understand how he purports to solve the problem, if such exists. He claims that an XML e-reader already exists (really? Even Wikipedia doesn't list one), that WYSIWYG XML editors are freely available (I wish, given that I use SCHEMA on my website), etc.

I don't know how he plans to force authors to write in XML, or word-processing/writing software companies to export in it, coherently and usably, or how he'll force the vendors to sell XML ebooks, particularly as ePUB is considered an XML-based format. Nobody sees the need for a new XML format, because ePUB IS the new XML format.

I really really officially give up on this thread, kids. I think it's reached my head-banging limits.

Hitch

[Hitch]

Sorry:

I just have one more thing I gotta say.

The whole "hoorah" about XML was simply this--that machines couldn't read HTML and figure out what each element was. THAT was the entire point of XML.

XML is meant to be read by machines.
HTML is meant to be read by humans.

We're discussing BOOKS here, not computer programs. The end user in mind is NOT a machine. It's a person.

Here endeth the lesson.

Hitch

[Toxaris]

Originally Posted by Hitch

And, now he's saying that he meant the colophon differently, so I'm completely lost. I don't understand what on earth he's talking about. He originally said, when mentioning colophon:



to which I replied, about DocBook, and now he says:

I think he means the part early in the book (or at the end at most ebooks nowadays) where stuff is placed like version, publisher, original title (in case of translations), dates, copyright info and so on.

I usually create a class called 'colophon' for that. Easy enough to find and more than enough 'semantics'.

[Hitch]

Originally Posted by Toxaris

I think he means the part early in the book (or at the end at most ebooks nowadays) where stuff is placed like version, publisher, original title (in case of translations), dates, copyright info and so on.

I usually create a class called 'colophon' for that. Easy enough to find and more than enough 'semantics'.

Ah-hah. After reading through this whole thing, again (about colophon, I mean), the light begins to dawn, overall. To answer you, though, Tox:

No, I mean..I know what a colophon in a print book IS; I just don't understand how the information in a print colophon has bupkus to do with what we were discussing. I'd said:

which assumes arguendo that the book creators would TAG that element for you.

(About the tagging of elements correctly, or at all.)

He replied to that thusly:

It is always 'tagged' in colophon; it would be in proper metadata if such existed.

Which left me pretty much where I was a moment ago, bewildered. I assumed he'd meant the Colophon SCHEMA,which kind of made sense, if we were discussing ye olden DocBook.

But then, when I'd mentioned DocBook-Colophon, that's when he came back and said:

No, the colophon in a physical (e-)book.

Which, if he's actually discussing ePUB formatting, makes no sense at all. I was talking about actual elements, like...images and paragraphs, blockquotes, and the like.

Thus, he is OBVIOUSLY only talking about metadata. This is why, @Tox, @eSchwartz, we all keep going round and round in circles. He's confusing the format of ePUB with whether or not Bookmaker Bob is tagging the book with metadata. It has nothing to do with the realities of the ePUB format itself. The fact that those two things are unrelated, realistically, seem to not be known to him. As an ePUB-maker himself, which he says he is, he ought to know that the semantics and meta available under the DC is fairly extensive. I can only assume that whatever process he's using isn't exposing him to those options.

Otherwise, the bit about how the colophon of a book is "always" tagged makes no sense. (I'd also note: most publishers no longer use Colophons. It's vestigial. Most simply have a logo and the copyright assertion. So...make of that what you will. 98% of all "colophons" would be utterly useless for his purposes.)

He's thinking that the METADATA and SEMANTIC tagging that he wants will all be provided in the Colophon, which obviously has no relationship whatsoever to the FORMAT of the book itself, whether in print, PDF, ePDF, ePUB, MOBI or Bob's Big Book Format.

This also explains why he's so enamored of such a rigid format. His rigidity is about the OCD-ness of qualifying this and that, so you can sort it, find it, etc. He doesn't really give two hoots about whether or not a chapter is called "chapter" in XML, or not, in XHTML. He's right--for his purposes, XML would, for the categorization of his completed/purchased books, be better. It is, after all, essentially a database for text-based data.

So, to create a database of his books, like in the Calibre Catalogue, he'd likely be better off with an XML list, of those books. (Is this a good point to mention that you can export an XML Catalogue list from Calibre?) XML formatting for his list--not XML formatting for the books themselves.

He's simply wrong about his issue being the format. BUT, you can tell from his posts--and his issue--that he's not the kind of guy that's going to change his mind.

I'm sticking with my penultimate post, however. XML is for machines, and HTML is for humans. Period.

Hitch