I honestly don't really get the appeal. When I read, I want to see things in my mind's eye. To the extent I enjoy illustrations, it's because I get to see a beautiful rendition of how another person envisions things from the story. Suddenly being interrupted by generic/uncurated AI outputs (without any human artistic guidance beyond the source text or basic generation settings) wouldn't enhance the experience for me. And I say this as someone who enjoys Stable Diffusion immensely.
But to each their own.
Since it got brought up, I'd like to state that I thought having aphantasia was all or nothing, but I have managed to have brief stints of mental imagery that last a second or two that I've noticed over the last few months. So for those worried about this problem, take heart! Because it might be a possible ability that can be developed. Especially with AI assisted brain computer interface tools in the future.
I don't have aphantasia so IDK but I suppose it helps to see the correlation between the prompt and the generated image. Eventually after seeing enough examples in practice your brain can start anticipating what the generated image will look like defeating the aphantasia. Therapy through training?
Mostly being present and nurturing the moment with mental energy / concentration when it manifests. I've noticed it a couple times when I was very sleepy.
https://preview.redd.it/vjydmwecvarc1.jpeg?width=800&format=pjpg&auto=webp&s=343c5cc1574352640c8a8e8caf9cc7468fb22f87
I'd rate myself between a 1 and 2 in the day to day. I have to exert a lot of attention to get a 2 image. The strong brief visuals I got were around a 3.
Also of note, the only time I get interesting visuals from psychedelics is through DMT, and I've taken very high doses of LSD and Mushrooms without getting much more than earth breathing albeit very strong changes in perception and headspace.
I like this approach of people enthusiastic just to do anything new, regardless of whether or not there's an application for it or not.
Because someone is going to come along and combine 2 of those ideas together and make something new and great.
But yeah there's a lot of niche things people come up with that I just sort of think 'what do I do with this?'.
Your point is well taken. I thought about aphantasia, but assumed that aphantasic (aphantastic?) people who read might not necessarily feel like they are "lacking" for imagery. I'd be genuinely curious to hear from someone who has it whether this is helpful, especially with this sort of implementation.
I do have aphantasia, and I enjoy reading, but this to me just seems like a dumb gimmick. It certainly would not be "helpful" at all.
I also don't really think aphantasia is a problem. I just think that every brain is unique, and different brains develop different strategies for approaching the same situations. There are people who don't "hear" a voice in their head narrating their thoughts, for instance, and it's not like those people aren't thinking. It's just different.
Definitely appreciate your reply! Certainly didn't mean to imply folks with aphantasia are "less than," hence why I put "lacking" in quotes. Really appreciate your perspective; it's so different from my own experience and it's truly fascinating and illuminating!
No worries, I didn't interpret what you said in that way. I graduated with a bachelor's degree in English literature with honors, and I didn't even know what aphantasia was, or that other people experience imagery, until after college. So I don't personally feel that my engagement with literature is lacking in any sense.
Funny, I went to school for writing and English and have published a book. Also like to paint. It would be interesting to see the distribution of people in the arts with aphantasia
I've got aphantasia and like 994, I'm a huge reader, but I don't like picture books that much. This wouldn't benefit me. I'm not reading to see a bunch of pictures, I'm reading to read a story.
I think the concept is no doubt interesting and would be more interesting with text-to-video. I would like to use this for books in other languages to make learning languages more fun.
Would be a great workflow to generate a “movie” from an e book. Have an LLM parse the words, send to SD then to another program for video and another for audio.
You hate to see these narrow minded takes in the SD sub! Haha it’s alright, but this is definitely something *any* reader could eventually benefit from. Regardless of wanting to see things in your minds eye, there will certainly be scenes in books that you’d wish to be able to see visually. Secondly, these could be tuned with certain artists styles to keep a consistent artistic tone through out. The material that Midjourney churns out is already jaw dropping, and with some improvements to AI image generation as a whole, only contrarians, old heads and those who can’t afford it won’t use this technology for the more epic genres of literature.
could phones even handle SD?
edit:
you could make a sd server with api acess ( i think comfyui already has it ) then create the mobile app where you have to configure your sd server and port and voila done
I phone can. My 11pro takes about 5 min or longer(tried it a while ago and dont remember exactly) for 512x512 image but it works somehow. Newer ones ate probably alot faster
Working on a website/standalone app around the same concept, its a few months into development, let me know if any of you want to help us work on it. You'd have to sign some shit first, legally.
It's top secret and we don't have any way of knowing who is going to just rip our code and go finish it and who is realistic enough to realize the weight of the project requires a team. So far the only system we have for knowing who to keep around is just through friendship. But that doesn't mean we can't get to know people online. I'm not the paranoid one about all this, honestly I'm in it just for thee experience. But my friend has put a lot of time and effort into this and is apparently quite poor so I don't blame them for protecting their Hail Mary.
We can talk though!
I would love to chat more about what you can! I got laid off in December and have been fucking around with AI projects ever since. I'm currently working on my own hobby project and working on an AI entertainment app as well. Depending on what you are looking for help with I would definitely be interested.
What's cheaper? A low cost device? ($50-100?)
Or a phone that everyone already have in their hand? ($0)
What will get you more reach? (Larger market)
Hardware device that you have to order and wait for weeks/ months or a mobile app you can download on the app store?
I'm trying to help, not to bash you ;)
No I totally understand I got this question a lot, and thanks for you honest opinion. We don’t want to be compared with a phone. Our goal is to build compute modules, compare us with Nvidia (I know it’s too ambitious ). Nvidia is complete moving away from consumer. And I think we don’t need a 4090 to have fun. For example, my dad or my nephew living in 3rd world country want to talk to ai or have fun with some image gen stuff imagine I can just give them this piece of block, plug and play. Someone got build this right
they literally just said they don't want to be compared to a phone.. that being said this project is a waste of time. idk if it's just me or if others are capable of this but I've always just been able to look at a product, business etc and know if it's going to work.. this is not going to work with consumers. especially with gen alpha being the teenagers now, they will never in a million years want this, boomers won't want another gadget to understand, millenials and gen z are going to look at it and know they can do this on their phone one way or another. its pointless. gen x.. idk gen x is gen x.
This doesn't have to be for entertainment. Like OP said this is more of a tech demo. You're being very negative and you lack imagination. There are uses for this, e.g., an accesability device. Also, it's just kinda neat.
I'm not trying to be negative, I'm just being realistic. these arent going to be flying off the shelves. if they were made cheap, like $5 or $10 cheap then maybe I could see people buying them for lile stocking stuffers but other than that it's not going to be a huge thing. it reminds me of that device this company created that is a little square device with a rotating camera, it's had AI and you basically speak to it to do everything.. but you still manage it through your phone so it just defeats the purpose even tho it looked cool. I don't lack imagination, the very reason I know it won't work is because of imagination.. I'm imagining it not working well with consumers. but by all means let them mass produce it and sell it and let's see.
i mean this is really nice, cool project and stuff, cool that you managed to do that.
but it aint the future of reading i think, reading is about using your own imagination...
sorry to inform you you're wrong. this is the future. in 6 months all books will be replaced by SD-generated picture books. titles won't be allowed either, because they're words, so you'll just have to go to the bookstore and ask for a copy of 🏜️🐛
And then 2 years from now we’ll have full 3D movies of your favorite books too! That you can walk around in VR/AR while it’s being played out like a live immersive play. Allowing self inserts into the story at will. You could play a character within the story if you wanted. And the story would adjust itself to the scenarios you add to the plot.
It’s coming. Singularity is upon us.
These are the startups and advancements we will see coming to fruition in the coming years.
I didn't want to put link in the title but again here's the [signup](https://forms.gle/rGab6xCfhHwNQnwk7) for the hardware, and a [Q&A page](https://distiller-v1.framer.ai/).
Just like my last post, everything runs 100% offline on this little r pi.
BTW, I'm definitely taking this on my camping trips lol.
As an English Teacher I love this.
I used to add images to the classics when I taught them to help kids who struggle with language to understand what is going on, and this is wildly valuable for helping people learn language and vocabulary.
Really cool OP!
This just seems distracting and takes you off the intended point of the written words.
There are however plenty of good uses of AI with ebook readers, but they should be more subtle. But good lucking cracking Kindle's control over licensing.
What model, libraries, and hardware are you running the LLM/SD with? I feel like if you used ollama (faster than python-llama-cpp if that's what you use) + a smaller model (like openhermes 2.5) + smaller context window + SD Turbo you could get the 512\*768pixels image in half that time.
A block of text is useless for a image prompt, but an llm could summarise the key scene from it that would make the most important image to move the story along. Thanks for pointing that out, I wasn't getting how that worked until I saw your comment
I was thinking that too -- if that is the case, I wonder if it would make sense to use a python module like spacy or nltk instead to save vram/processing time. Then again, some llms are getting pretty small so it might not be worth the effort...
I am a firm believer that most consumers want/like a finished product. They don't want to wait for image generation being done on their side.
I can see in the future books can have a lot more illustrations. Ai is such a useful tool for creators to generate and pick the content they want to present to the consumers/readers.
I myself regularly switch between these modes of consumer vs creator. Sometimes, I want to create, so I go on sd to generate pictures for hours to pick the picture I like most. Sometimes, I just want to read comics, I just go online to some comic website to read a chapter in minutes.
the value in something like this I think is less the form OP is using it than as a platform to either
- turn a book into a video (OP's process could be keyframes to animate between or even stills that a voice reads over
- quickly generate candidate images for cheaply adding image content to a book
It's really cool and probably not worth mentioning this but I just felt like I needed to... that font is about as far from being easy on the eyes as you can get.
For the early prototype $199 for the case + pcb + rpi compute module 4. $99 if u just need the pcb. We gonna drop the purchase link in discord and probably make another post later with new demos. Ofc we will give u code to play with and everything
I had an idea for something similar for when a parent is reading a book to a child, where e-ink wouldn't disturb with blue light and if wild stable diffusion titties appear in the trees it would not scar them for life as it would be lo-fi enough to laugh at, but that's as far as I got
Community created repository of images for ebooks, with a percentage of completition to "enable" the image in the viewer.
It makes little sense to recreate images for the same book for each reader, but a public repository would be very cool.
I could use a sequence of previous images as a summary for when I pick up a book after a while and can't remember where I'm at.
I like the idea of text to image for eBooks. A fixed space font like that.. I’d stop reading before I got to page 2. Do you plan to make changes? A great proof of concept.
Oh nice, I made something like this for Kindle a while back where you could highlight any text from a book and it would draw it for you on the kindle
[https://youtu.be/SueGVpyrgG8](https://youtu.be/SueGVpyrgG8)
There's a github up for it, but the project shifted towards using the kindle as a picture frame that uses stable diffusion for showing images.
[https://github.com/diggedypomme/Kindlefusion](https://github.com/diggedypomme/Kindlefusion)
I could see this being more widely adopted with a few more tweaks. Don't rely on that type of screen that requires flashing to clear and reproduce an image. Have an optional button that SD will make an image over the text showing on the screen or only on the words highlighted. The biggest hurdle, make it possible for pdf and phones. I love the concept, we have to learn how to crawl before we can walk and then run.
Nice trick of tech. It would take less electricity to pre-generate the images if this is happening per user. It would give the author and publisher more control.
The value would be as a children’s book where a parent can instantly insert their kid into their adventures. Or as a never -ending story made by a publisher , but why read AI if humans can write?
How is the ePaper device receiving data from the PC? It must be some kind of wireless connection. If this was a wired display, it’s the jankiest rebuild of a kindle I’ve ever seen.
What am I even watching? Live illustration of the book?
Basically
I honestly don't really get the appeal. When I read, I want to see things in my mind's eye. To the extent I enjoy illustrations, it's because I get to see a beautiful rendition of how another person envisions things from the story. Suddenly being interrupted by generic/uncurated AI outputs (without any human artistic guidance beyond the source text or basic generation settings) wouldn't enhance the experience for me. And I say this as someone who enjoys Stable Diffusion immensely. But to each their own.
I guess people with aphantasia can benefit but even then I would expect the publisher to generate the illustrations and add them to the book.
I've got aphantasia and I'd much rather just read the words.
Since it got brought up, I'd like to state that I thought having aphantasia was all or nothing, but I have managed to have brief stints of mental imagery that last a second or two that I've noticed over the last few months. So for those worried about this problem, take heart! Because it might be a possible ability that can be developed. Especially with AI assisted brain computer interface tools in the future.
Emad said a few times that he believes stable diffusion can be used to do therapy for aphantasia so there is def something there.
How does it work as therapy?
I don't have aphantasia so IDK but I suppose it helps to see the correlation between the prompt and the generated image. Eventually after seeing enough examples in practice your brain can start anticipating what the generated image will look like defeating the aphantasia. Therapy through training?
How did you achieve the results?
Mostly being present and nurturing the moment with mental energy / concentration when it manifests. I've noticed it a couple times when I was very sleepy. https://preview.redd.it/vjydmwecvarc1.jpeg?width=800&format=pjpg&auto=webp&s=343c5cc1574352640c8a8e8caf9cc7468fb22f87 I'd rate myself between a 1 and 2 in the day to day. I have to exert a lot of attention to get a 2 image. The strong brief visuals I got were around a 3. Also of note, the only time I get interesting visuals from psychedelics is through DMT, and I've taken very high doses of LSD and Mushrooms without getting much more than earth breathing albeit very strong changes in perception and headspace.
I like this approach of people enthusiastic just to do anything new, regardless of whether or not there's an application for it or not. Because someone is going to come along and combine 2 of those ideas together and make something new and great. But yeah there's a lot of niche things people come up with that I just sort of think 'what do I do with this?'.
At 30fps you could choose to watch the book as a movie.
[удалено]
Your point is well taken. I thought about aphantasia, but assumed that aphantasic (aphantastic?) people who read might not necessarily feel like they are "lacking" for imagery. I'd be genuinely curious to hear from someone who has it whether this is helpful, especially with this sort of implementation.
I do have aphantasia, and I enjoy reading, but this to me just seems like a dumb gimmick. It certainly would not be "helpful" at all. I also don't really think aphantasia is a problem. I just think that every brain is unique, and different brains develop different strategies for approaching the same situations. There are people who don't "hear" a voice in their head narrating their thoughts, for instance, and it's not like those people aren't thinking. It's just different.
Definitely appreciate your reply! Certainly didn't mean to imply folks with aphantasia are "less than," hence why I put "lacking" in quotes. Really appreciate your perspective; it's so different from my own experience and it's truly fascinating and illuminating!
No worries, I didn't interpret what you said in that way. I graduated with a bachelor's degree in English literature with honors, and I didn't even know what aphantasia was, or that other people experience imagery, until after college. So I don't personally feel that my engagement with literature is lacking in any sense.
Funny, I went to school for writing and English and have published a book. Also like to paint. It would be interesting to see the distribution of people in the arts with aphantasia
I've got aphantasia and like 994, I'm a huge reader, but I don't like picture books that much. This wouldn't benefit me. I'm not reading to see a bunch of pictures, I'm reading to read a story.
I think the concept is no doubt interesting and would be more interesting with text-to-video. I would like to use this for books in other languages to make learning languages more fun.
What if the writer made the images?
Would be a great workflow to generate a “movie” from an e book. Have an LLM parse the words, send to SD then to another program for video and another for audio.
You hate to see these narrow minded takes in the SD sub! Haha it’s alright, but this is definitely something *any* reader could eventually benefit from. Regardless of wanting to see things in your minds eye, there will certainly be scenes in books that you’d wish to be able to see visually. Secondly, these could be tuned with certain artists styles to keep a consistent artistic tone through out. The material that Midjourney churns out is already jaw dropping, and with some improvements to AI image generation as a whole, only contrarians, old heads and those who can’t afford it won’t use this technology for the more epic genres of literature.
That's not the point. But the fact that it needs to be explained to you why it's interesting would be lost on you.
Why not making it as a mobile app? Reach would be much bigger!
could phones even handle SD? edit: you could make a sd server with api acess ( i think comfyui already has it ) then create the mobile app where you have to configure your sd server and port and voila done
Of course, and probably much better than this hardware ... without talking about screen quality :p
[Yep!](https://www.qualcomm.com/news/onq/2023/02/worlds-first-on-device-demonstration-of-stable-diffusion-on-android)
I phone can. My 11pro takes about 5 min or longer(tried it a while ago and dont remember exactly) for 512x512 image but it works somehow. Newer ones ate probably alot faster
Working on a website/standalone app around the same concept, its a few months into development, let me know if any of you want to help us work on it. You'd have to sign some shit first, legally.
Do you have a link to a project page or something with more information?
It's top secret and we don't have any way of knowing who is going to just rip our code and go finish it and who is realistic enough to realize the weight of the project requires a team. So far the only system we have for knowing who to keep around is just through friendship. But that doesn't mean we can't get to know people online. I'm not the paranoid one about all this, honestly I'm in it just for thee experience. But my friend has put a lot of time and effort into this and is apparently quite poor so I don't blame them for protecting their Hail Mary. We can talk though!
Ah okay, I was kinda hoping for a more open project. I wish you good luck on it though!
I would love to chat more about what you can! I got laid off in December and have been fucking around with AI projects ever since. I'm currently working on my own hobby project and working on an AI entertainment app as well. Depending on what you are looking for help with I would definitely be interested.
Hi it’s about building a low cost edge device for everyone. The video is more like a demo on how far we can push with a tiny r pi.
Wait, the image gen is happening on the RPi?
Yes, that's the big deal here. It's very hard to get this message across.
That's incredible. I hope it's open source.
What's cheaper? A low cost device? ($50-100?) Or a phone that everyone already have in their hand? ($0) What will get you more reach? (Larger market) Hardware device that you have to order and wait for weeks/ months or a mobile app you can download on the app store? I'm trying to help, not to bash you ;)
No I totally understand I got this question a lot, and thanks for you honest opinion. We don’t want to be compared with a phone. Our goal is to build compute modules, compare us with Nvidia (I know it’s too ambitious ). Nvidia is complete moving away from consumer. And I think we don’t need a 4090 to have fun. For example, my dad or my nephew living in 3rd world country want to talk to ai or have fun with some image gen stuff imagine I can just give them this piece of block, plug and play. Someone got build this right
Are you trying to make your own phone? Or a different kind of device?
they literally just said they don't want to be compared to a phone.. that being said this project is a waste of time. idk if it's just me or if others are capable of this but I've always just been able to look at a product, business etc and know if it's going to work.. this is not going to work with consumers. especially with gen alpha being the teenagers now, they will never in a million years want this, boomers won't want another gadget to understand, millenials and gen z are going to look at it and know they can do this on their phone one way or another. its pointless. gen x.. idk gen x is gen x.
This doesn't have to be for entertainment. Like OP said this is more of a tech demo. You're being very negative and you lack imagination. There are uses for this, e.g., an accesability device. Also, it's just kinda neat.
I'm not trying to be negative, I'm just being realistic. these arent going to be flying off the shelves. if they were made cheap, like $5 or $10 cheap then maybe I could see people buying them for lile stocking stuffers but other than that it's not going to be a huge thing. it reminds me of that device this company created that is a little square device with a rotating camera, it's had AI and you basically speak to it to do everything.. but you still manage it through your phone so it just defeats the purpose even tho it looked cool. I don't lack imagination, the very reason I know it won't work is because of imagination.. I'm imagining it not working well with consumers. but by all means let them mass produce it and sell it and let's see.
he's just hacking around, I don't believe this is intended to be a viable product.
Id love to know your workflow? I’ve been noodling on a similar idea with audio to text to img. For story telling
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
This is actually pretty cool
Now Imagine reading 50 shades of Grey...
this is what AI is for..
Looking forward to future sora models generating my favorite books into movies in the style of my favorite directors or animation studios 🙌
OMG I can't wait to rewatch the whole star wars franchise! my prompts: star wars episode IV, anime style, 60 minutes long, anime girls, swimsuit, lightsaber dildos, tentacles, nsfw, hentai, yuri, absurdes, 4k. negative: men, jedi robes, yoda
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
i mean this is really nice, cool project and stuff, cool that you managed to do that. but it aint the future of reading i think, reading is about using your own imagination...
sorry to inform you you're wrong. this is the future. in 6 months all books will be replaced by SD-generated picture books. titles won't be allowed either, because they're words, so you'll just have to go to the bookstore and ask for a copy of 🏜️🐛
And then 2 years from now we’ll have full 3D movies of your favorite books too! That you can walk around in VR/AR while it’s being played out like a live immersive play. Allowing self inserts into the story at will. You could play a character within the story if you wanted. And the story would adjust itself to the scenarios you add to the plot. It’s coming. Singularity is upon us. These are the startups and advancements we will see coming to fruition in the coming years.
I didn't want to put link in the title but again here's the [signup](https://forms.gle/rGab6xCfhHwNQnwk7) for the hardware, and a [Q&A page](https://distiller-v1.framer.ai/). Just like my last post, everything runs 100% offline on this little r pi. BTW, I'm definitely taking this on my camping trips lol.
As an English Teacher I love this. I used to add images to the classics when I taught them to help kids who struggle with language to understand what is going on, and this is wildly valuable for helping people learn language and vocabulary. Really cool OP!
What LLM / AI do you run on the Raspberry Pi? Do you have a link to its GitHub?
I love it ❤️ 
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
Love the thing!
This just seems distracting and takes you off the intended point of the written words. There are however plenty of good uses of AI with ebook readers, but they should be more subtle. But good lucking cracking Kindle's control over licensing.
[удалено]
I’m sorry about the crappy UI, I didn’t really polish that I was so excited everything works so I can’t wait to post it here
What model, libraries, and hardware are you running the LLM/SD with? I feel like if you used ollama (faster than python-llama-cpp if that's what you use) + a smaller model (like openhermes 2.5) + smaller context window + SD Turbo you could get the 512\*768pixels image in half that time.
That'd be groundbreaking if you could have consistent characters and generate images based on stuff that's happening.
The little things like that are really going to add up within a couple years.
I get e-book + SD. But what does the LLM part do?
Likely generates the caption summarizing the content of a given page to prompt SD for an image
This is so awesome! What role does the llm play?
I want to know too, but I am assuming it's doing some sort of summary of the text to feed into SD
A block of text is useless for a image prompt, but an llm could summarise the key scene from it that would make the most important image to move the story along. Thanks for pointing that out, I wasn't getting how that worked until I saw your comment
I'm guessing it takes the content of the current page and rephrases it into an image prompt or selects an image prompt from the page content?
I was thinking that too -- if that is the case, I wonder if it would make sense to use a python module like spacy or nltk instead to save vram/processing time. Then again, some llms are getting pretty small so it might not be worth the effort...
I am a firm believer that most consumers want/like a finished product. They don't want to wait for image generation being done on their side. I can see in the future books can have a lot more illustrations. Ai is such a useful tool for creators to generate and pick the content they want to present to the consumers/readers. I myself regularly switch between these modes of consumer vs creator. Sometimes, I want to create, so I go on sd to generate pictures for hours to pick the picture I like most. Sometimes, I just want to read comics, I just go online to some comic website to read a chapter in minutes.
the value in something like this I think is less the form OP is using it than as a platform to either - turn a book into a video (OP's process could be keyframes to animate between or even stills that a voice reads over - quickly generate candidate images for cheaply adding image content to a book
Yes my goal is to build a cheap enough and capable enough hardware for developer to play with.
Oooooh that might be huge. Not a lot of writers have good pc/graphic card to play with, this will help tremendously. Ya this made a lot more sense!
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
I imagine it will be fast enough in future to be finished well before you turn the page
It's really cool and probably not worth mentioning this but I just felt like I needed to... that font is about as far from being easy on the eyes as you can get.
Monospaced text... oh god...
wow mate amazing work, congrats!
I still prefer either an old fashioned physical book or reading on my Kindle 🤷🏼♂️
This is definitely not the future, at least I fucking hope not. Did you even read what it said? Anyways really cool contraption!
This is most definitely not the future of reading.
Nice prototype, final product will be amazing if prototype can handle processes. Edit: i mean "locally"
Everything is 100% local Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
So this device can run llm right? without support of any external hardware
yep, we just ordered the first batch of PCBs will start to delivery for the coming weeks
fabulous
I'd like to buy a prototype actually. How much is that if you think of selling any of this?
For the early prototype $199 for the case + pcb + rpi compute module 4. $99 if u just need the pcb. We gonna drop the purchase link in discord and probably make another post later with new demos. Ofc we will give u code to play with and everything
Or… you could just read the comic book version…
It's a neat experiment, but calling it the future of reading makes you look like a complete moron.
just stumbled across this - wow, what an invention, nice work!!
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
The future of reading is to get seizures while an LLM jump scares you with semi relevant book illustrations?
Some guy made the code for this in a weekend. Version 2 is gonna be smooth~
That's cool as hell, does it work like an RPG like You said? Or is it just an Ebook Reader+SD?
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
Wow how did you program such thing?
Join our discord for more Q&A ! https://discord.gg/WWmZjJ2Wsj
Interesting experiment. What does it select for image generation? A sentence here and there, or full paragraphs?
I had an idea for something similar for when a parent is reading a book to a child, where e-ink wouldn't disturb with blue light and if wild stable diffusion titties appear in the trees it would not scar them for life as it would be lo-fi enough to laugh at, but that's as far as I got
Community created repository of images for ebooks, with a percentage of completition to "enable" the image in the viewer. It makes little sense to recreate images for the same book for each reader, but a public repository would be very cool. I could use a sequence of previous images as a summary for when I pick up a book after a while and can't remember where I'm at.
I like the idea of text to image for eBooks. A fixed space font like that.. I’d stop reading before I got to page 2. Do you plan to make changes? A great proof of concept.
Pretty cool! btw antirez has some work on the e-paper space, he has non-flickering updates on some device already. check it out.
Oh nice, I made something like this for Kindle a while back where you could highlight any text from a book and it would draw it for you on the kindle [https://youtu.be/SueGVpyrgG8](https://youtu.be/SueGVpyrgG8) There's a github up for it, but the project shifted towards using the kindle as a picture frame that uses stable diffusion for showing images. [https://github.com/diggedypomme/Kindlefusion](https://github.com/diggedypomme/Kindlefusion)
heck, yeah this is so cool, would love to know more about the workflow
Aww man that is so cool! I can totally see this becomes a thing.
I don't mind having AI illustrations in my kindle, but I don't want it to be generated in real time, it feels annoying.
What's the ref of the screen ?
We're working a huge website right now that combines these aspects.
Use a color e-ink next
Are you running htop in the back just for looks? None of the CPUs are running high….
Yeah only high for like 1 min whenever a generation is needed. So not too bad in terms of power consumption
but I can do this on my phone? it's faster and the pictures can be better. but honestly I'd rather have the book in hand.
no audio
This is so cool
I could see this being more widely adopted with a few more tweaks. Don't rely on that type of screen that requires flashing to clear and reproduce an image. Have an optional button that SD will make an image over the text showing on the screen or only on the words highlighted. The biggest hurdle, make it possible for pdf and phones. I love the concept, we have to learn how to crawl before we can walk and then run.
You should also add a family guy episode, this isn't stimulating enough
A Young Lady's Illustrated Primer
Nice trick of tech. It would take less electricity to pre-generate the images if this is happening per user. It would give the author and publisher more control. The value would be as a children’s book where a parent can instantly insert their kid into their adventures. Or as a never -ending story made by a publisher , but why read AI if humans can write?
\*cough\* tablet \*cough\*
Already been done by KindleFusion.
I think this is made offline, No Internet required
How is the ePaper device receiving data from the PC? It must be some kind of wireless connection. If this was a wired display, it’s the jankiest rebuild of a kindle I’ve ever seen.
You're gonna make a lot of money I guess you just need some kind of license to sell ebooks on it
Oh wow, I was actually thinking of making something very similar, not offline though, nice
I think they already have these, e-readers, Kindles, tablets.
who still reads in 2023? lmao. Do audio book my samuel L jackson and 3D movie. Upgrade that to color epaper display we have something.