Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic Lua Scripting Support #1671

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open

Conversation

NPO-197
Copy link

@NPO-197 NPO-197 commented Apr 20, 2023

Here is a proposed starting point to adding Lua script support for MelonDS natively. Currently has bare minimal functionality, as I would like to have others review my work before making to many additions. This is my first project in c++ so please let me know if there are any errors, or parts that I need to re-do. The aim for this is to support accessibility features and trackers, not as a comprehensive debugging tool.

@NPO-197 NPO-197 marked this pull request as ready for review April 21, 2023 11:57
@nadiaholmquist nadiaholmquist force-pushed the master branch 3 times, most recently from e6c2fb6 to df0ab90 Compare April 23, 2023 16:51
Copy link
Member

@RSDuck RSDuck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The LuaThread is a bit dubious and probably needs some thoughts how this could be implemented better.

src/frontend/qt_sdl/OSD.cpp Outdated Show resolved Hide resolved
luaThread = new LuaThread();
connect(luaThread,&LuaThread::signalPrint,console,&LuaConsole::onGetText);
connect(luaThread,&LuaThread::signalClearConsole,console,&LuaConsole::onClear);
connect(buttonPausePlay,&QPushButton::clicked,luaThread,&LuaThread::luaTogglePause);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

placement of spacing here and in a bunch of other places.

src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
LuaScript.md Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
@NPO-197
Copy link
Author

NPO-197 commented Apr 28, 2023

Still some more changes needed but majority of the refactoring is done, no more LuaThread, the Lua state is created and run by the emuThread. This should hopefully address any concerns around potential race conditions. I plan on cleaning up other issues on Sunday.

@NPO-197
Copy link
Author

NPO-197 commented May 6, 2023

I think this should be good for another review, I believe I have addressed all the major issues that have been brought up. If there is anything else that I need to change or fix please let me know. There are still more features / functions I'd like to add, but this is the kind of project that could easily lead to feature creep, so I'm gonna cut myself off here.

src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.h Outdated Show resolved Hide resolved
src/frontend/qt_sdl/OSD.cpp Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.h Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
src/frontend/qt_sdl/main.h Outdated Show resolved Hide resolved
src/frontend/qt_sdl/main.h Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
@RSDuck
Copy link
Member

RSDuck commented May 6, 2023

there's probably still some more things, but this is what I found from a glance.

@NPO-197
Copy link
Author

NPO-197 commented May 7, 2023

OK addressed those issues, still some bugs I found that I need to fix, I plan on working on that tomorrow.

@NPO-197
Copy link
Author

NPO-197 commented May 23, 2023

OK ready for another review @RSDuck when you have time.

Copy link
Member

@RSDuck RSDuck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to go over all the changes and fix the spacing. Then once the merge conflicts have been solved I guess we can merge it.

LuaScript.md Outdated Show resolved Hide resolved
@NPO-197
Copy link
Author

NPO-197 commented Jun 8, 2023

OK went over everything, double checked to make sure the spacing was good. Fixed spelling errors in the markdown. Will probably need someone else to take a look at the merge conflicts at least for the .yml and CMakeLists files.

@RSDuck
Copy link
Member

RSDuck commented Jun 12, 2023

blah something went wrong while resolving the merge conflict.

@RSDuck
Copy link
Member

RSDuck commented Jun 12, 2023

hm I don't really understand how dependencies are managed on macOS, I can't find anything on how it's installed in the code.

@RSDuck
Copy link
Member

RSDuck commented Jun 19, 2024

To the person who talked about this PR on our IRC:

If you are reading this, we are very sorry, the ban was due to the anti spam bot bot for some reason flagging your messages. Let us know how you can communicate again with us.

@Arisotura
Copy link
Member

adding on Generic's message: I removed the ban, so you can come back to the IRC. sorry about it.

@NPO-197
Copy link
Author

NPO-197 commented Jun 27, 2024

Hey sorry I have been kind of a ghost for a while! It's been over a year since I started this fork, I'm not super up to date on all the changes since then. I'd like to get this project working again with the current version, since I am able to continue working on it now.
I don't have much experience with merging repo's so I will start by asking:
Is there a way to update this fork so it's not like 100+ commits behind main, or should I try and make a new fork from the current version?
Also I wanted to thank everyone for all the help and feedback so far! I know this isn't exactly a high priority feature, but it's been a wonderful learning experience for me personally, and hopefully others will find it helpful in the future.

@RSDuck
Copy link
Member

RSDuck commented Jun 27, 2024

there have been a ton of changes on the frontend especially with regards to multi instance within one process support. At this point it would probably smarter to not rebase/merge and instead start fresh while moving over the more independent parts like the Lua interface.

@NPO-197
Copy link
Author

NPO-197 commented Jun 28, 2024

Understood! I'd prefer that option honestly. Most of the work was on the Lua interpreter side of things, I tried to keep the front end stuff to a minimum while working on this since I figured I'd probably need to remake the front end entirely at some point. So I won't be loosing much.

Also gives me a chance to refresh my memory on how everything works.

I'll make a new fork later today, once that's up I'll add a link to the new one and close this pull request.

@jahndan
Copy link

jahndan commented Jul 9, 2024

adding on Generic's message: I removed the ban, so you can come back to the IRC. sorry about it.

Hello, sorry I just saw this. I don't really know how to use IRC, and I just saw you guys added a link to the discord so I'll probably stick to that since it's more familiar.

I actually started a very rudimentary merge of this branch to main. (It definitely doesn't work, I was just using the merge conflicts to track which parts of the code base I hadn't yet looked at.) Because I've been busy IRL, I haven't touched anything in about 2 weeks, but before that I was mainly trying to get an overview of all the changes made externally to the Lua scripting module itself (i.e. what kind of integration was even necessary to begin with) as well as trying to understand the general structure of the codebase (and also what pieces got refactored, which were many).

Also gives me a chance to refresh my memory on how everything works.
I'll make a new fork later today, once that's up I'll add a link to the new one and close this pull request.

I don't see the new fork yet on your profile, but I'll push the half-merge (and some of my notes) I've got so far to mine if you want to take a look (I was in the middle of figuring out how/where to associate a Lua scripting engine for each instance, before starting to refactor the Lua module as a class that could have multiple instances). my fork with the partial merge

@NPO-197
Copy link
Author

NPO-197 commented Jul 10, 2024

I'll be sure to take a look at your notes, although I think for right now my goal is simply build that can get a helloworld.lua file to run... Once I have that then I'll start working on re-adding all the important features...

Although yes, one of the first things that will need to be figured out is how to handle multiple instances, since that is the biggest change from back when I first started.

I should be able to work on it some today, I originally wanted to start on this like 2 weeks ago but my availability has been... to keep it simple, rather unpredictable as of recently. Hopefully I will have the time to get a good start this week, however, just so everyone is aware, my progress will likely remain somewhat scattered for now.

I'll be sure to post a link to new fork here once I can get, at the bare minimum, "hello world" working on a single instance so we can have a starting point. In the meantime feel free to ask me any questions!

Also thanks again to everyone for the help and patience.

@ag-advania
Copy link

Hello, any update on this project ? Thanks !

@NPO-197
Copy link
Author

NPO-197 commented Aug 30, 2024

Small update for those interested, (thanks to everyone who reached out about this)
Currently I'm waiting on getting a new PC before I continue working on this project, should be able to get started again by the end of September, by then I should have much more free time to continue working on this, and eventually get this project to a state where it can be merged with the main project. Thanks again to everyone's support and assistance, as well as everyone's patience.

For those who have reached out asking if they can help in any way, I only ask for a little more time to get a minimal viable version working that is compatible with the recent updates to melonDS. Once that basic frame work is setup, I will be sure to let everyone know, and will then gladly appreciate any help!

@gaithern
Copy link

Hype!

@NPO-197
Copy link
Author

NPO-197 commented Oct 18, 2024

Finished testing on Linux and Mac using a VM, everything seems to run as expected on Linux, had to change the default font for one of the drawing functions, since it seemed to not work on Mac. I wrote up a simple Lua script and placed it under tools to test all the different functions, everything seems to be working on all platforms!

With that finished that's everything on my to-do list crossed off, (sans better documentation) so really all that's needed now is to get someone else to give it a look over and let me know what needs to be improved / changed.

@edo9300
Copy link

edo9300 commented Oct 18, 2024

Finished testing on Linux and Mac using a VM, everything seems to run as expected on Linux, had to change the default font for one of the drawing functions, since it seemed to not work on Mac. I wrote up a simple Lua script and placed it under tools to test all the different functions, everything seems to be working on all platforms!

With that finished that's everything on my to-do list crossed off, (sans better documentation) so really all that's needed now is to get someone else to give it a look over and let me know what needs to be improved / changed.

There's another gotcha that i noticed and you should be wary about (and is present in yoru code), lua errors in a c function will corrupt the stack if there are objects with constructors present on the stack frame because lua (unless compiled as c++) uses longjmp/setjmp to handle script errors. This is not an issue on windows because on there longjmp also calls destructors, but is an issue on every other platform. In this pr the only function exhibiting this issue is drawImage

QString path = luaL_checklstring(L,1,NULL);
int x = luaL_checkinteger(L,2);
int y = luaL_checkinteger(L,3);
int sx = luaL_optinteger(L,4,0);
int sy = luaL_optinteger(L,5,0);
int sw = luaL_optinteger(L,6,-1);
int sh = luaL_optinteger(L,7,-1);

If you pass a valid string but invalid other arguments, that QString won't be properly destroyed, so you should move its creation after all the lua_check functions have been executed (since in your case you're not calling anything else that would call into lua and possibly raise lua errors)

@NPO-197
Copy link
Author

NPO-197 commented Oct 20, 2024

@edo9300 The last push should have resolved the issue you pointed out.

I also took a look at using a C++ compiled version of the lua library, but as you already mentioned on discord since we aren't shipping it as a statically linked library we can't assume that a C++ compiled version of lua will always be available...

There may be a way to setup Cmake to compile lua from source ourselves, or something to that effect, but I don't really think that the benefits of doing so would outweigh the added complexity, at least not currently.

Created a basic Documentation of currently implemented lua functions.
@NPO-197
Copy link
Author

NPO-197 commented Oct 22, 2024

Finished writing up the documentation for the currently implemented Lua functions.

@RSDuck
Copy link
Member

RSDuck commented Oct 22, 2024

Please make the use of whitespaces consistent with the rest of the code.

Round one of clean up
Round 1-2 of code clean up
Pass 1-3 should be ready for another round of review.
@NPO-197
Copy link
Author

NPO-197 commented Oct 23, 2024

Please make the use of whitespaces consistent with the rest of the code.

Sorry about that, thanks for the reminder @RSDuck!
Did a second pass-through and cleaned up everything to be inline with CONTRIBUTING.md / clean up extraneous whitespaces.

Let me know if there is anything else I need to go back and take a look at! 😄

Renamed "MelonPrint" to just "print",
TIL you can redefine pretty much any lua funct...
Also fixed the lua console not re-opening after closing it the first time.
Switched running the LUA script from the emuThread to the main window / main GUI thread... Should have just done this from the start, this makes things much simpler to implement, and will make future planned GUI functions much easier to add.
Made for loop more readable in Screen.cpp
Removed include for LuaMain in EmuThread as it is no longer needed.
Cleaned up some whitespace / old commented out code.
@NPO-197
Copy link
Author

NPO-197 commented Nov 1, 2024

With that, I should be ready for another review. If @RSDuck or someone has time this weekend to look over it, I'd appreciate it!

Copy link
Member

@RSDuck RSDuck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still lots of space missing.

We will unfortunately not merge this before the coming 1.0 release, as there are enough new things already which might explode.

src/frontend/qt_sdl/EmuInstance.h Outdated Show resolved Hide resolved
src/frontend/qt_sdl/Screen.cpp Outdated Show resolved Hide resolved
src/frontend/qt_sdl/LuaMain.cpp Outdated Show resolved Hide resolved
@gaithern
Copy link

gaithern commented Nov 4, 2024

Is there any way we can get a compiled version of this branch in the releases page on this repo in the meantime?

@AntonioND
Copy link
Contributor

Also, could you upload some examples of how to use it?

@NPO-197
Copy link
Author

NPO-197 commented Nov 5, 2024

@gaithern @AntonioND , made a test release of the current build, you can find the current documentation under the ./tools/LuaScripts/ folder here. let me know if you have any trouble!

As of now the currently implemented Lua functions are still rather bare bones, most of the work so far has been just setting up the foundation. If there are specific features / functions that you would find useful to have that aren't implemented yet, then let me know and I can prioritize adding those features next.

I agree with RSDuck about waiting until after the 1.0 release before we think about merging this onto main, It simply doesn't make sense to worry about something that's still very much in the experimental phase like this with such a major release on the horizon.

That being said I do plan to continue working on this in the mean time. 😄
Thanks as always for everyone's help! 💛

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants