Updated 2017-03-02 21:43:24 by bll

BallroomDJ, by bll, is a ballroom music player written in tcl/tk.

Translators needed: https://crowdin.com/project/ballroomdj/invite

bll 2014-2-7 ; updated 2017-3-1

Wanting a gui portable to many operating systems (why do people use anything other than pack and grid?), I immediately turned to Tcl/Tk.

BallroomDJ provides interfaces to manage the songs, edit the songs, play those songs from a manually selected list or automatically based on various rules, and optional audio file organization.

Several external programs are used: vlc for playback, python-mutagen for mp3 tag handling, metaflac for flac tag handling, SoX to modify the audio files and various volume controls for linux, windows and mac osx.

BallroomDJ requires a scrolling window for upwards of 30000 songs. I initially wrote a scrolling frame, but it only worked for about 1k-2k rows of data (as each row has some widgets in it). I wrote Virtual Scrolling to handle this amount of data, as none of the other solutions such as scrolling canvases or scrolling frames work with large amounts of data.

The database is generally kept entirely in memory. I changed BallroomDJ to use a random access file interface for the database to allow individual entries in the database to be updated. This is not super fast, but works well and allows intercommunication of the database changes between the different processes, allowing for a much better user experience. The random access file is space padded and has a header and trailer so I can still source it to read it in (as the tcl parser is much faster than anything I can write in tcl). I still have to process and convert the dictionary entries in the database to objects, as objects are not serializable.

While I don't have any translations as yet other than English(US) and English(GB), the program is fully localizable. I recently upgraded the comboboxes (Combobox Enhancements) to better support localizations. All buttons have had any -width specifier removed. Unicode filenames are interesting. python-mutagen now fully supports them. For other programs like SoX that do not, I have to rename the files, do the processing and then rename the file back to its proper unicode name again. Mac OS X has decomposed unicode filenames. These have to be converted with iconv (or tcllib has routines to do it).

The user interface is very difficult to get working well for everyone without getting messy or cluttered. The font scaling works great with 4K displays, but I still had to go through my code and find places where I sized by pixels and fix those spots.

2016-1-9 : I believe that all of the localization issues have been resolved now that collation has been implemented. All the by-pixel sizing (spacing and borders) has been changed to points so high resolution displays should not have any sizing issues once font scaling is set.

2016-7-11 : Version 3.0: After reading BOOK About Face (still not finished), I started the project of (a) getting rid of my multitude of windows, (b) modeless validation and (c) more intuitive interfaces where you simply do your work and don't have to manage the work flow and save all the time.

Removing the multitude of windows presented some issues. I briefly thought about keeping the programs as external executables and simply embedding their windows into another frame. This idea was abandoned as it more of a hack and not a solution.

Instead I decided to put each program's window into a notebook tab in the main program. The first step was placing each external program into its own namespace. Then when an external program is needed, it is source'd. I have a little source manager module that tracks whether a program has been sourced already and if so, simply calls its main procedure. These external programs that are now in their own namespace still feel like separate entities, even though they are sourced and are really part of the same program.

The prior version used sockets for all the inter-process communication and that hasn't changed. It may be a bit unusual to use sockets to communicate with what is essentially the same process. But that was a huge rewrite I didn't want to do. It has the advantages in that the event driven code still works the same way and also the external programs can still be used as separate executables if needed.

The one major socket problem was when a program sent a message and needed to get a response. Since the program is now monolithic, it can't both wait for a response and process the sent message at the same time. There is now a 'sendget' routine that will call the receiver's socket handler proc directly with a special socket name. The receiver's socket handler calls the socket send routine as usual to send the response and the socket module checks for the special socket name and saves the response data in a variable. After the receiver's socket handler exits, the 'sendget' routine can now continue and pulls the response data out of the variable.

Rather than have each executable destroy all of its ui widgets and restart from scratch each time (since destroying and creating widgets is slow), the widgets are only built once when the program is first sourced. Where I had context sensitive ui elements, I had to always create the widgets and the startup code determines whether the widget is gridded or removed.

Many of the variables that were initialized by default caused a lot of problems and I had to search through the code and make sure those variables were explicitly initialized or unset, otherwise restarting the executable (which was external and now sourced) picked up old data. I had fallen out of a good habit of always explicitly initializing every variable and that made the rewrite more difficult. Where it was reasonable, I tried to have the executable keep the data that was last used so when it was restarted, it was essentially where the user left off.

Since the program is now essentially monolithic, I was able to make the database a singleton object. This saves on a lot of database loads and will speed up the user's work.

Menus turned out to be a problem. My program uses context sensitive menus everywhere. Destroying and replacing the menu caused horrible flicker on all platforms. And Linux has a bug where the replaced menu will not appear in a maximized window. Instead, I have only one menu, and it destroys all of its current entries and rebuilds the new entries from the menu associated with the current notebook tab.

Almost every dialog is now contained in a new notebook tab or if there was room and it made sense, simply added to the existing notebook tab. My program has very few toplevel windows now. It is a lot cleaner and easier to use. When a notebook tab is closed it will call a registered "close handler" for the tab. When the user switches to a different notebook tab, an optional "save handler" is called.

Since the main window is now maximized by default, I added an internal window with a size grab bar in some places so that the view size could be changed by the user. This view size is automatically saved but turned out to be a little tricky to restore the same view size.

2017-3-1 : Version 3.7.x: The volume controls have been converted to loadable dynamic libraries (instead of exec'ing a volume program). This reduces the resource usage and reduces volume latency.

The VLC player has also been integrated as a dynamic library. The libVLC interface provides a much better interface to VLC than using a socket. It is possible to determine exactly when the music starts playing, and the initial seek when the "song start" is non-zero is now very reliable. I was not able to call Tcl procedures from the libVLC events, as that would crash (probably due to VLC threads), but the state changes can be recorded.

Startup on windows is getting rather slow due to the wish/tclsh restart and the number of dynamic libraries that are loaded.

After converting the external processes to internal processes in version 3.0, the socket communication between processes was still present. The internal communications have been converted to use reflected channels. Reflected channels act like FIFOs rather than as multiple channels as a socket interface does. To send a message and get a response, a second out-of-band reflected channel is used, otherwise messages to the normal FIFO will get picked up. For some reason, I cannot close the reflected channels without crashing Tcl/Tk, so there is some minor resource leakage there.

sbron's dbus-tcl package has been put to use to support MPRIS enabled players on Linux. BallroomDJ can now use many different Linux players, though I suspect that MPRIS implementation differences and playlist handling differences may be a problem.

I added an update feature to BallroomDJ, and in the process found that http is unreliable (version 8.6.6). Instead, I use the 'wget' program.

References: [Windows XP: volume control] [Windows: volume control] [Mac OS X: volume control] [Linux: volume control] [VLC Tcl Extension]