jMonkeyEngine.org

Cool, I suggest making a separate audio renderer for this purpose. Since its never easy syncing the audio samplerate with dynamic video I think it might be easier to go that way instead of presuming that the output occurs in a certain moment according to computations on the passed time. How do you do the syncing now? Do you adjust the audio output later to the video or do you play the audio with varying samplerate? Do you get drift with long sequences?

Almost forgot — there’s two patches to JME3 to use these projects.

Index: src/desktop/com/jme3/system/JmeSystem.java
===================================================================
--- src/desktop/com/jme3/system/JmeSystem.java	(revision 8529)
+++ src/desktop/com/jme3/system/JmeSystem.java	(working copy)
@@ -299,7 +299,10 @@
                 clazz = (Class<? extends AudioRenderer>) Class.forName("com.jme3.audio.lwjgl.LwjglAudioRenderer");
             } else if (settings.getAudioRenderer().startsWith("JOAL")) {
                 clazz = (Class<? extends AudioRenderer>) Class.forName("com.jme3.audio.joal.JoalAudioRenderer");
-            } else {
+            } else if (settings.getAudioRenderer().equals("Send")){
+            	clazz = (Class<? extends AudioRenderer>) Class.forName("com.aurellem.capture.audio.AudioSendRenderer");
+            }
+            else {
                 throw new UnsupportedOperationException(
                         "Unrecognizable audio renderer specified: "
                         + settings.getAudioRenderer());

Index: src/desktop/com/jme3/system/Natives.java
===================================================================
--- src/desktop/com/jme3/system/Natives.java	(revision 8529)
+++ src/desktop/com/jme3/system/Natives.java	(working copy)
@@ -122,7 +122,11 @@
         extractNativeLib(sysName, name, load, true);
     }

-    protected static void extractNativeLib(String sysName, String name, boolean load, boolean warning) throws IOException {
+    protected static void extractNativeLib(String sysName, String name, boolean load, boolean warning) throws IOException{
+    	extractNativeLib(sysName, name, load, warning, false);
+    }
+
+    protected static void extractNativeLib(String sysName, String name, boolean load, boolean warning, boolean force) throws IOException {
         String fullname = System.mapLibraryName(name);

         String path = "native/" + sysName + "/" + fullname;
@@ -148,7 +152,7 @@
                 long sourceLastModified = conn.getLastModified();

                 // Allow ~1 second range for OSes that only support low precision
-                if (targetLastModified + 1000 > sourceLastModified) {
+                if ((!force) && (targetLastModified + 1000 > sourceLastModified)) {
                     logger.log(Level.FINE, "Not copying library {0}. Latest already extracted.", fullname);
                     return;
                 }
@@ -194,6 +198,7 @@
         String audioRenderer = settings.getAudioRenderer();
         boolean needLWJGL = false;
         boolean needOAL = false;
+        boolean needAudioSend = false;
         boolean needJInput = false;
         boolean needNativeBullet = isUsingNativeBullet();
         if (renderer != null) {
@@ -206,6 +211,10 @@
                 needLWJGL = true;
                 needOAL = true;
             }
+            else if (audioRenderer.equals("Send")) {
+            	needLWJGL = true;
+            	needAudioSend = true;
+            }
         }
         needJInput = settings.useJoysticks();

@@ -223,8 +232,11 @@
                     extractNativeLib("windows", "lwjgl64");
                 }
                 if (needOAL) {
-                    extractNativeLib("windows", "OpenAL64");
+                    extractNativeLib("windows", "OpenAL64", false, true, true);
                 }
+                if (needAudioSend){
+                	extractNativeLib("windows/audioSend", "OpenAL64", true, true, true);
+                }
                 if (needJInput) {
                     extractNativeLib("windows", "jinput-dx8_64");
                     extractNativeLib("windows", "jinput-raw_64");
@@ -238,8 +250,11 @@
                     extractNativeLib("windows", "lwjgl");
                 }
                 if (needOAL) {
-                    extractNativeLib("windows", "OpenAL32");
+                    extractNativeLib("windows", "OpenAL32", false, true, true);
                 }
+                if (needAudioSend){
+                	extractNativeLib("windows/audioSend", "OpenAL32", true, true, true);
+                }
                 if (needJInput) {
                     extractNativeLib("windows", "jinput-dx8");
                     extractNativeLib("windows", "jinput-raw");
@@ -256,8 +271,11 @@
                     extractNativeLib("linux", "jinput-linux64");
                 }
                 if (needOAL) {
-                    extractNativeLib("linux", "openal64");
+                    extractNativeLib("linux", "openal64", false, true, true);
                 }
+                if (needAudioSend){
+                	extractNativeLib("linux/audioSend", "openal64", true, true, true);
+                }
                 if (needNativeBullet) {
                     extractNativeLib("linux", "bulletjme", true, false);
                 }
@@ -270,8 +288,11 @@
                     extractNativeLib("linux", "jinput-linux");
                 }
                 if (needOAL) {
-                    extractNativeLib("linux", "openal");
+                    extractNativeLib("linux", "openal", false, true, true);
                 }
+                if (needAudioSend){
+                	extractNativeLib("linux/audioSend", "openal", true, true, true);
+                }
                 if (needNativeBullet) {
                     extractNativeLib("linux", "bulletjme", true, false);
                 }
@@ -285,6 +306,9 @@
                 }
 //                if (needOAL)
 //                    extractNativeLib("macosx", "openal");
+                if (needAudioSend){
+                	extractNativeLib("macosx/audioSend", "openal", true, true, true);
+                }
                 if (needJInput) {
                     extractNativeLib("macosx", "jinput-osx");
                 }

sincerely,
–Robert McIntyre

I see. Well cool, then I’d suggest just adding a set of default output settings that work well (like 25fps/30fps with 44.100Hz/48.000Hz etc.). For the SDK a “Development Application Harness” to facilitate application-SDK integration is planned anyway, it could easily incorporate a switch for this video output renderer.

We’d make it a plugin for the SDK, it contains all the jars etc needed.. It would not be part of the “core” engine I guess, because as you say its probably a bit oversized for that.

It can make anything that Xuggle can, including mkv, ogg, flv, etc.

The other two writers are fallbacks if Xuggle is missing. They don’t include any patent encumbered codecs, so there should be no problems with including the library in any apps. It does not require the xuggle jar files to function.

The simplest and most compatible option is to use the FileVideoRecorder class and output a sequence of PNG files, which can be encoded into whatever format is desired.

Since it outputs audio and video as separate files, some amount of post processing is necessary to make the final video.

I’ve been reading the documentation for creating a jMonkeyEngine3 SDK plug-in and have four questions:

1.) Is creating the plug-in something I would do myself and then make available here, or is it something that a core developer would make based on the three library jars?

2.) How can I make sure my native code gets compiled for all the platforms that JME supports? I see something that looks like a multi-platform automated build system here: https://www.newdawnsoftware.com/jenkins/. Is this part of JME/is it an appropriate place for my native code?

3.) If the recording system is released as a SDK plug-in, how will do people who don’t use the SDK get access to it?

4.) Does it make sense to make it a plug-in, or would it be better to just provide the jars somewhere, possibly distributed with JME, along with lots of documentation on the wiki?

Thanks for your clarifications — I’ll work on making plug-in for the SDK.

How does jMonkeyEngine3 produce the jME3-lwjgl-natives.jar file if there’s no automatic way to compile native code for multiple systems?

sincerely,
–Robert McIntyre