Author |
Topic: Sysex bug (Read 49165 times) |
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
Using Sonar 4 pentium 4 2.4 with 1gig ram
Using MIDI-OX and Yoke.
Input USB Turtle beach midi port
routed to Sonar 4
Output sonar four is Yoke 1
Input to Midi-OX is Yoke 1
Output to Korg M1 is Usb Turtle beach midi port
I have sysex files for my M1 stored in sonar 4
They work fine without MIDI-OX
but after hooking it up I can't successfully send a correct file.
There seems to be extreme data corruption.
So I compared the input and output windows of MIDIOX
and wow...
Sysex from Sonar 4 in the input window is different than
the output window...
I would think the input and output would be identical.
obviously some manipulation of sysex is going on??
Steve 
|
| « Last Edit: Nov 2nd, 2005, 3:10pm by Troub » |
 IP Logged |
|
|
|
Jamie OConnell
Administrator
 
Gender: 
Posts: 2027
|
 |
Re: Sysex bug
« Reply #1 on: Nov 3rd, 2005, 3:23pm » |
 Quote  Modify
|
on Nov 2nd, 2005, 3:00pm, Troub wrote:
Sysex from Sonar 4 in the input window is different than
the output window...
|
|
In what way is it different? You can capture the SysEx output in a 2nd MIDI-OX instance if you attach it to the 1st instance via MIDI Yoke 2. In the 2nd Instance, Choose View |SysEx... Then SysEx | Receive Manual Dump... Then start the dump in SONAR.
|
|
IP Logged |
--Jamie
Music is its own reward.
|
|
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
|
Re: Sysex bug
« Reply #2 on: Nov 3rd, 2005, 11:56pm » |
Quote Modify
|
I have the following setup:
Korg M1 >> USB Turtle Beach input
USB Turtle Beach Input >> Sonar 4 IN
Sonar 4 Out >> Yoke 1
Yoke 1 >> MIDI-OX In
MIDI-OX Out >> USB Turtle Beach Output
USB Turtle Beach Out >> Korg M1.
When I dump from Sonar 4 the sysex goes out to the Korg and tries to load but there are data corruption errors.
When I look at the input and output windows in MIDI-OX I see the same data coming in and going out but The differences are isolated internal corruptions.
The length's are the same roughly but the internal Hex codes are only similar. They match for 32 bytes or so then some garbage is thrown in... then they match again for awhile.
They are most different at the begining of the dump and more similar at the end but by that time everything is offset incorrectly for the Korg to read correctly.
I assumed from the help files that I could just put MIDI-OX on each side of SONAR to monitor input and output to and from SONAR... I was trying the output side first.
OF COURSE, the sysex dump to the Korg works fine untill I put Midi-OX into the mix.
Is it possible to contact you directly and discuss?
Steve
|
|
IP Logged |
|
|
|
Jamie OConnell
Administrator
Gender:
Posts: 2027
|
|
Re: Sysex bug
« Reply #3 on: Nov 4th, 2005, 11:52pm » |
Quote Modify
|
Quote:| They match for 32 bytes or so then some garbage is thrown in... then they match again for awhile. |
|
What type of garbage? Is it illegal SysEx data (outside the range of 00 - 7F) or just packaged differently from the input buffers? MIDI-OX might package the data differently when it is sending it to the output port. MIDI Yoke often will package data differently as it passes it along. It should always be the same data in the same order, but there are no rules as to how much or how little data should be placed into a SysEx buffer (obviously it shouldn't overflow). This certainly is true when input buffers are sized differently from output buffers as well. Differing MIDI Interface driver manufacturers will package SysEx messages differently as well.
If you have some short examples of mismatched input vs. output data feel free to post it in a message here. You can copy data from either monitor to the clipboard and then paste it into a message.
You definitely should be able to place different instances of MIDI-OX on either side of SONAR.
Sorry, but my current schedule does not permit time even for paid consulting. This forum is an attempt to provide at least some help to people (plus others may benefit).
|
| « Last Edit: Nov 4th, 2005, 11:55pm by Jamie OConnell » |
IP Logged |
--Jamie
Music is its own reward.
|
|
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
|
Re: Sysex bug
« Reply #4 on: Nov 9th, 2005, 12:33am » |
Quote Modify
|
The Data is in the proper sysex range but in hex it's actually different. I could understand if it were just nulls that were interjected due to packaging and timing issues but the data is actually altered, and of course my M1 won't accept the data when it has come through the MidiOX. I'll post some samples in the next one but they're LARGE so I'll try and get excerpts that START at the same location and SHOULD be the same. I don't really NEED help or MIDIOX. just thought that it was strange and you would be interested if it is a bug.
|
|
IP Logged |
|
|
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
|
Re: Sysex bug
« Reply #5 on: Nov 9th, 2005, 12:38am » |
Quote Modify
|
TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT
00013929 1 -- F0 Buffer: 8192 Bytes System Exclusive
SYSX: 00 63 63 35 00 22 00 01 00 00 00 00 00 00 00 00 00 00
SYSX: 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32
SYSX: 00 00 00 00 00 00 00 63 00 63 00 63 00 00 00 00 00 00
SYSX: 53 6F 00 66 74 20 48 6F 72 6E 10 73 00 00 5B 7F 00 00
SYSX: 00 00 00 00 60 40 00 00 00 60 40 00 00 00 03 00 00 00
SYSX: 04 02 00 09 00 0C 00 00 0B 00 28 28 14 14 00 00 00 1F
SYSX: 63 0D 03 00 00 0F 00 03 04 21 00 16 10 37 2E 00 00 7B
SYSX: 05 02 00 00 00 00 00 63 63 26 02 36 58 1E 00 00 63 02
SYSX: 00 63 2F 08 39 08 36 1C 00 32 36 00 41 00 00 05 00 63
SYSX: 54 54 55 53 0A 00 00 00 00 00 00 00 00 00 00 00 00 00
SYSX: 00 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SYSX: 32 00 00 00 00 00 00 00 63 00 00 63 00 63 00 00 00 00
SYSX: 00 00 48 65 6C 6C 73 42 00 65 6C 6C 73 01 00 29 01 7F
SYSX: 15 00 00 00 24 60 00 42 00 00 60 42 00 00 00 00 00 00
SYSX: 00 00 02 00 00 06 00 0C 00 0A 00 19 40 19 1E 1E 00 00
SYSX: 1F 34 00 00 50 0A 68 01 00 00 00 1E 00 28 37 2E 00 00
SYSX: 03 7B 1D 10 00 63 34 63 00 00 00 00 00 00 4A 00 00 00
SYSX: 00 00 63 63 63 63 00 63 63 00 40 00 00 00 00 00 00 00
SYSX: 63 0D 51 00 00 34 2F 00 00 00 00 00 10 00 00 00 62 71
SYSX: 00 00 04 00 42 37 4C 00 00 00 00 00 63 5E 63 58 00 50
SYSX: 00 63 40 00 00 00 00 00 00 00 63 58 00 00 00 4C 00 00
SYSX: 00 00 00 44 72 6F 00 70 20 20 20 20 20 20 08 00 20 34
SYSX: 7F 00 00 00 00 00 00 7C 41 00 00 60 00 41 00 00 00 00
SYSX: 00 00 00 00 02 12 03 00 21 00 00 06 02 0A 0A 13 13 00
SYSX: 08 00 3F 04 48 00 00 00 40 00 0B 0C 3A 00 00 27 00 63
SYSX: 00 00 00 00 59 00 00 00 00 00 00 00 00 75 00 00 63 00
SYSX: 63 62 00 63 00 63 63 63 00 52 00 59 00 3E 00 55 00 63
SYSX: 00 63 00 4F 00 50 00 01 00 41 01 08 01 00 00 00 00 00
SYSX: 00 00 00 00 63 00 00 00 00 00 00 00 00 00 00 00 00 00
SYSX: 00 00 00 32 00 00 00 00 00 00 00 63 00 63 00 00 63 00
SYSX: 40 40 40 00 00 5A 65 70 68 79 72 20 40 20 20 00 01 00
SYSX: 2E 7F 00 1F 00 00 00 15 60 3D 00 00 00 60 3D 00 00 00
SYSX: 00 00 00 00 00 02 00 06 00 00 07 00 02 17 32 32 00 32
SYSX: 32 00 00 1F 21 00 12 14 2A 45 00 7F 00 4E 00 00 00 00
|
|
IP Logged |
|
|
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
|
Re: Sysex bug
« Reply #6 on: Nov 9th, 2005, 12:38am » |
Quote Modify
|
Ok, Now from the Output window
TIMESTAMP IN PORT STATUS DATA1 DATA2 CHAN NOTE EVENT
00013929 1 1 F0 Buffer: 8192 Bytes System Exclusive
SYSX: 66 06 54 4A 60 6C 06 22 6C 7A 08 02 60 6C 64 04 64 78
SYSX: 11 50 03 72 78 50 05 4A 33 04 56 02 60 04 64 02 66 64
SYSX: 10 7E 03 56 10 7E 6C 07 72 10 40 03 22 12 1D 7A 0B 10
SYSX: 18 72 04 56 33 1C 62 05 4A 1C 40 04 67 22 20 7A 03 56
SYSX: 28 7C 4C 04 56 28 66 03 72 28 1D 58 05 22 30 74 02 56
SYSX: 3B 34 50 04 22 38 72 02 02 56 40 70 00 00 00 60 28 00
SYSX: 00 00 3C 0D 10 18 55 3C 07 10 24 3C 06 10 2A 30 3C 12
SYSX: 10 48 3C 05 55 10 54 3C 04 10 60 3C 6A 12 10 78 3C 04
SYSX: 10 04 5D 3C 04 10 10 3C 10 10 3B 28 3C 06 10 34 3C 04
SYSX: 03 10 40 70 00 00 F7 00 00 00 00 00 00 00 00 00 00 00
SYSX: 00 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SYSX: 32 00 00 00 00 00 00 00 63 00 00 63 00 63 00 00 00 00
SYSX: 00 00 48 65 6C 6C 73 42 00 65 6C 6C 73 01 00 29 01 7F
SYSX: 15 00 00 00 24 60 00 42 00 00 60 42 00 00 00 00 00 00
SYSX: 00 00 02 00 00 06 00 0C 00 0A 00 19 40 19 1E 1E 00 00
SYSX: 1F 34 00 00 50 0A 68 01 00 00 00 1E 00 28 37 2E 00 00
SYSX: 03 7B 1D 10 00 63 34 63 00 00 00 00 00 00 4A 00 00 00
SYSX: 00 00 63 63 63 63 00 63 63 00 40 00 00 00 00 00 00 00
SYSX: 63 0D 51 00 00 34 2F 00 00 00 00 00 10 00 00 00 62 71
SYSX: 00 00 04 00 42 37 4C 00 00 00 00 00 63 5E 63 58 00 50
SYSX: 00 63 40 00 00 00 00 00 00 00 63 58 00 00 00 4C 00 00
SYSX: 00 00 00 44 72 6F 00 70 20 20 20 20 20 20 08 00 20 34
SYSX: 7F 00 00 00 00 00 00 7C 41 00 00 60 00 41 00 00 00 00
SYSX: 00 00 00 00 02 12 03 00 21 00 00 06 02 0A 0A 13 13 00
SYSX: 08 00 3F 04 48 00 00 00 40 00 0B 0C 3A 00 00 27 00 63
SYSX: 00 00 00 00 59 00 00 00 00 00 00 00 00 75 00 00 63 00
SYSX: 63 62 00 63 00 63 63 63 00 52 00 59 00 3E 00 55 00 63
SYSX: 00 63 00 4F 00 50 00 01 00 41 01 08 01 00 00 00 00 00
SYSX: 00 00 00 00 63 00 00 00 00 00 00 00 00 00 00 00 00 00
SYSX: 00 00 00 32 00 00 00 00 00 00 00 63 00 63 00 00 63 00
SYSX: 40 40 40 00 00 5A 65 70 68 79 72 20 40 20 20 00 01 00
SYSX: 2E 7F 00 1F 00 00 00 15 60 3D 00 00 00 60 3D 00 00 00
SYSX: 00 00 00 00 00 02 00 06 00 00 07 00 02 17 32 32 00 32
SYSX: 32 00 00 1F 21 00 12 14 2A 45 00 7F 00 4E 00 00 00 00
SYSX: 00 00 03 00 01 64 39 00 00 39 00 08 00 00 63 3C 00 00
SYSX: 00 00 00 00 00 63 59 00 00 00 00 63 00 37 48 00 1C 00
|
|
IP Logged |
|
|
|
Jamie OConnell
Administrator
Gender:
Posts: 2027
|
|
Re: Sysex bug
« Reply #7 on: Nov 9th, 2005, 3:40pm » |
Quote Modify
|
Nulls definitely mean something in SysEx and cannot be inserted or sorted without changing meaning.
As you say, theses are excerpts and not the full dumps. So they are taken rather out of context. The second one is longer than the first but neither are the full 8192 bytes. The second one shows an imbedded F7 followed by something other than F0, so that is indicative of a problem of some sort. I'd have to see the full dumps to really get any sort of handle on what's going on, but my guess is that there is some unexpected data in the input buffers that is confusing MIDI-OX.
|
| « Last Edit: Nov 9th, 2005, 3:42pm by Jamie OConnell » |
IP Logged |
--Jamie
Music is its own reward.
|
|
|
Troub
New Member
MIDI-OX Rules!
Posts: 6
|
|
Re: Sysex bug
« Reply #8 on: Nov 9th, 2005, 8:42pm » |
Quote Modify
|
These were both starting at the beginning, not out of context. I just grabbed the first section of both. However they are too long to post here... they are part of a 50-60KByte Total system load for an M1.
Nothing prior...
I can email you complete dumps if you're curious. Just let me know where to send them.
|
| « Last Edit: Nov 9th, 2005, 8:45pm by Troub » |
IP Logged |
|
|
|
jwpgroup
Member
MIDI-OX Rules!
Gender:
Posts: 12
|
|
Re: Sysex bug
« Reply #9 on: Dec 11th, 2006, 6:54am » |
Quote Modify
|
I'm pretty sure that there is a bug with Sysex and Midi-USB (Midi-OX 7.0.0.365). The problem manifiests itself as a data corruption in the outgoing data stream which inserts a '00' every 240 bytes followed by the next two bytes being shifted and the third 'lost'. Choosing packets of 240 bytes in length shows just this corruption. Packets of a different length result in data loss as well.
We are currently working as a third-party developer coding Midi-USB routines and have used Midi-OX as part of our testing (it's great BTW!). This bug has taken us a couple of days to pin down as we where convinced it must be our software/hardware - but it isn't - it's Midi-OX.
We finally got to the bottom of the problem by sending a file from Midi-OX and crosslinking the real world Midi I/O on our product (so the file would return to another version of Midi-OX). We thought it was our hardware (so we checked further), but the same data corruption problem shows with two other midi-usb converters we have here (from other manufacturers), so our hardware is fine.
To finish the test we then sent the file using Sysex97 (a simple utility we found on the web) and the outgoing dump was not corrupted in any way. QED (unfortunatly) Midi-OX corrupts outgoing USB-Midi Sysex data.
Take a Midi-USB converter and couple the input to the output and then send a Sysex file to see the issue.
Let me know if I can help with further diagnosis.
John
|
|
IP Logged |
|
|
|
JerryJorgenrud
New Member
MIDI-OX Rules!
Posts: 0
|
|
Re: Sysex bug
« Reply #10 on: Dec 12th, 2006, 2:18pm » |
Quote Modify
|
Here's how I test these sorts of things.
MIDI-OX:1 -> hardware out A -> MIDI Cable -> Hardware In B -> MIDI-OX:2
Open the Sysex view in OX:2 and select Sysex/Receive Manual Dump.
Open the Sysex view in OX:1 and select File/Send Sysex File.
Back in OX:2 when the file is finished I OK the manual dump box and the received sysex appears in the Display Window. In the Command window I load the same sysex file. If the byte count is not the same then something is wrong. If the byte count is the same then I run Sysex/Compare Windows to make sure the bytes are identical and not just the same count.
If there seems to be an issue I'll couple OX:1 to OX:2 with MIDIYoke and run the same tests. If MIDI-OX has an issue it should show up there as well as with a hardware port.
For an even more stressful test I'll use only one instance of MIDI-OX so it is both sending and receiving at the same time.
I do this with a variety of test sysex files, from short 7 byte files to large files in excess of 100k. Some of the files have multiple sysex strings inside and some are single sysex. The largest single sysex is about 35k. The largest file takes about 10 minutes to make the round trip.
To test your issue I used a USB MIDI port with output cabled to input. I usually run 256 byte buffers but you felt 240 was a magic number so I tried that as well as the 256 byte ones. I tried running with and without a delay after F7 and with a delay between buffers in settings ranging from 1 to 60 milliseconds. (Less than 1 and my interface chokes.) I also tried using only one instance of MIDI-OX.
All of the files were sent and received without incident.
You don't mention that you're using MIDIYoke somewhere in your setup but if you are you should upgrade to the newest version just released. It fixed a rare sysex corruption issue.
Otherwise I'm hard-pressed to offer any suggestions ...
|
|
IP Logged |
|
|
|
jwpgroup
Member
MIDI-OX Rules!
Gender:
Posts: 12
|
|
Re: Sysex bug
« Reply #11 on: Dec 13th, 2006, 5:12pm » |
Quote Modify
|
Hi Jerry
thanks for that.
This scenario...
"MIDI-OX:1 -> hardware out A -> MIDI Cable -> Hardware In B -> MIDI-OX:2 Open the Sysex view in OX:2 and select Sysex/Receive Manual Dump".
Is exactly what I've done to prove the problem (no other item in the chain), except I'm using external USB-MIDI devices. I'm 100% positive it's a bug as I've now tested it with our hardware, a 'T' box USB-Midi converter and a M-audio MIDIsport. They all fail in exactly the same way with a file sent from MIDI-OX and are completly fine with the same file via Sysex97.
TBH, I wasn't prepared to believe it at first as the company we are working for (a well known name in the MIDI interface field) where convinced they had use MIDI-OX for similar transfers in the past. However... after further discussions on this specific point we concluded that they had used the programme with a computer with a internal plug-in midi port and not over USB, so that must be where the problem lies.
"If there seems to be an issue"...
Trust me, this is worth investigation. Three independent class compliant devices (on XP) from different companies can't all be wrong and fail (in the same way), where a different sysex Tx programme (sysex97) has no problem.
The buffer situation... "I usually run 256 byte buffers but you felt 240 was a magic number"... is only relevent to the loss of data. The first corruption occurs at 240 bytes in (& then on) and sending data in chunks of this length seems to stop bytes getting 'lost' at the corruption point (the file is still corrupted though). Just so you know, with buffers the same as the corruption length the timing delay can be zero and you still get all the file (well you do on our kit & the 'T' box - the MIDIsport seems to need time to think!).
As covered above, I'm not using anything but MIDI-OX and a single channel converter with the output coupled to the input. Bear in mind that this can be one of three different makes (two of which are on the market) , so I don't think it's a hardware issue.
Here is the first bit of the file....
Sent first;
F0 00 20 13 0B 7F 6E 18 70 1F 65 18 70 1F 5D 65 18 70 1F 65 18 70 3B 1F 65 18 70 1F 65 00 76 00 20 61 18 70 1F 65 0E 18 70 1F 65 28 06 00 22 40 40 00 00 40 64 00 00 00 40 48 01 00 40 70 00 01 00 40 00 00 00 00 10 08 01 00 40 18 01 00 22 40 40 01 00 40 60 01 00 00 40 00 02 00 40 20 00 02 00 40 40 02 00 40 10 60 02 00 40 00 02 00 22 40 20 02 00 40 40 02 04 00 40 60 02 00 40 00 00 03 00 40 20 03 00 40 00 40 03 00 40 60 03 00 22 40 00 03 00 40 20 03 44 00 40 40 03 00 40 60 00 03 00 40 00 04 00 40 00 20 04 00 40 40 04 00 00 40 60 04 00 40 00 00 00 00 00 00 00 00 00 00 08 00 00 00 00 04 00 40 01 20 04 00 40 00 00 00 20 00 00 00 00 00 40 04 04 00 40 60 04 00 40 00 48 05 00 40 7F 5F 2D 69 08 00 10 4F 61 02 00 2D 59 69 4C 04 1F 65 4C 64 2B
Received;
F0 00 20 13 0B 7F 6E 18 70 1F 65 18 70 1F 5D 65 18 70 1F 65 18 70 3B 1F 65 18 70 1F 65 00 76 00 20 61 18 70 1F 65 0E 18 70 1F 65 28 06 00 22 40 40 00 00 40 64 00 00 00 40 48 01 00 40 70 00 01 00 40 00 00 00 00 10 08 01 00 40 18 01 00 22 40 40 01 00 40 60 01 00 00 40 00 02 00 40 20 00 02 00 40 40 02 00 40 10 60 02 00 40 00 02 00 22 40 20 02 00 40 40 02 04 00 40 60 02 00 40 00 00 03 00 40 20 03 00 40 00 40 03 00 40 60 03 00 22 40 00 03 00 40 20 03 44 00 40 40 03 00 40 60 00 03 00 40 00 04 00 40 00 20 04 00 40 40 04 00 00 40 60 04 00 40 00 00 00 00 00 00 00 00 00 00 08 00 00 00 00 04 00 40 01 20 04 00 40 00 00 00 20 00 00 00 00 00 40 04 04 00 40 60 04 00 40 00 48 05 00 40 7F 5F 2D 69 08 00 10 4F 61 02 00 2D 59 69 00 4C 04 65 4C 64 2B
The seventh byte from the end shows the corruption.
Hope this helps!
Let me know if I can assist further with the diagnosis.
All the best
John
|
|
IP Logged |
|
|
|
JerryJorgenrud
New Member
MIDI-OX Rules!
Posts: 0
|
|
Re: Sysex bug
« Reply #12 on: Dec 15th, 2006, 7:00pm » |
Quote Modify
|
Hi John,
To follow up, you said:
[I did everything you did] "except I'm using external USB-MIDI devices"
and
"they had used the programme with a computer with a internal plug-in midi port and not over USB, so that must be where the problem lies"
It seems you missed the part where I said:
"To test your issue I used a USB MIDI port"
In fact, like you, I used an M-Audio MIDISport. I don't see how it can be a USB issue anyway. Applications don't do anything different, they don't even know if the MIDI interface is USB, serial, PCI, joystick, virtual, whatever. I spent a couple more hours on three machines trying to get corrupted data out of the MIDISport but it just isn't happening.
I had never heard of SysEx97 but I found that and all I can say is that shows you how simple it really is to send a sysex file. I poked around a bit and noticed that he uses what I would call a large amount of large buffers -- 30 of them at 8192 bytes if I'm right. Maybe MIDI-OX would work better in your situation if you sized the buffers bigger.
You also might try just "Actions -> Send -> SysEx File" rather than loading the file up in SysEx View. That brings the complexity all the way down to ReadFile, MemCopy, here's your buffer.
I didn't grasp this at first but recently started to wonder if perhaps when you say "USB-MIDI" you mean the type of USB-MIDI that is only implemented on Win XP (partially) and that MS says you don't need to write your own USB drivers for? I suppose that's what you mean when you say "class compliant"?
There are eight black boxes between my side:
http://msdn2.microsoft.com/en-us/library/ms789663.aspx
and yours (scroll down)
http://msdn2.microsoft.com/en-us/library/ms789375.aspx
|
|
IP Logged |
|
|
|
jwpgroup
Member
MIDI-OX Rules!
Gender:
Posts: 12
|
|
Re: Sysex bug
« Reply #13 on: Dec 16th, 2006, 6:16am » |
Quote Modify
|
Hi Jerry
I'm honestly not trying to be antagonistic here 
The problem we have (which is 100% repeatable) was first raised by our customer during his beta testing (different setup, different computer, different location). He sees exactly what I see.
Between the two of us (we provided the USB-MIDI microcontroller code - compliant to the USB-MIDI 1.0 spec, he designed the hardware & wrote all the other code), it took us three days to establish that the problem (for both parties) lay in the sysex Tx from MIDI-OX (Rx is fine).
Initially, like you and our customer, I was not prepared to believe it was a MIDI-OX issue, because we all think it's really great (honestly). Hand on heart, we could not have done this dev work without this application. So... It had to be our stuff right - after all we where the ones with beta software.
Anyways eventually after a lot of testing and head scratching (& really as a last resort) in a moment of desperation (!) I tried the two other MIDI-USB things we have here (the M-Audio MIDIsport UNO and the Tbox). I really was astonished when they both failed in exactly the same way.
To then backup this testing (& get a working transfer), I searched the web and found sysex97. I completly agree that it's not anything compared to MIDI-OX, but at this point I needed something (pre-written) to do a sysex transfer as a test that our code was not the problem. It worked, so that's why I mention it.
So, let's look at this a bit more...
"In fact, like you, I used an M-Audio MIDISport."
I really don't understand why you are not seeing the failure we are. I'm running XP Pro SP2 on a Dell M60 laptop (in case hardware interaction is the issue). I don't know what our customer was using but I suspect it was a reasonably powered box.
"I don't see how it can be a USB issue anyway. Applications don't do anything different"
I only raised this because our customer was convinced he had done sysex transfers reliably in the past (with MIDI-OX). The only difference in our new scenario is the use of USB as opposed to a plug-in MIDI port.
"I had never heard of SysEx97 but I found that and all I can say is that shows you how simple it really is to send a sysex file."
AS I mentioned above, I'm not holding this up as perfect (far from it), solution. It does work here (& with our customer) though, where MIDI-OX doesn't. 
"Maybe MIDI-OX would work better in your situation if you sized the buffers bigger."
I tried all that and it made no difference. It doesn't explain why you don't see it with the same hardware.
"You also might try just "Actions -> Send -> SysEx File"
That is what I'm doing.
"I didn't grasp this at first but recently started to wonder if perhaps when you say "USB-MIDI" you mean the type of USB-MIDI that is only implemented on Win XP (partially) and that MS says you don't need to write your own USB drivers for? I suppose that's what you mean when you say "class compliant"?"
Boy (now we've worked on this) do I know why you say that like that  Our contract was to provide routines to enable a single-chip microcontroller to bi-directionally communicate MIDI over USB to a Windows XP and/or MAC OSX PC without any custom drivers in accordance with the USB Device Class Definition for MIDI devices 1.0. no more, no less.
It's a published ratified spec and we meet it (as does the MIDIsport UNO and the 'T' box). The client did not want a custom driver, they wanted an out of the box solution, which I have to say works really well (apart from this issue with MIDI-OX).
Clutching at straws here, I'll PM you the Sysex dump we are using to see if that fails with you. I'd be happy to discuss this on the phone or do a log-me-in so you can see the problem.
Let me know if you'd like to take this further.
Thanks again
John
|
|
IP Logged |
|
|
|
JerryJorgenrud
New Member
MIDI-OX Rules!
Posts: 0
|
|
Re: Sysex bug
« Reply #14 on: Dec 16th, 2006, 11:50am » |
Quote Modify
|
" I'm honestly not trying to be antagonistic here"
Well neither am I. There's nothing like a good bug. The sysex code has been undisturbed for so long moths fly out when I open the files.
"I'm not holding this up as perfect (far from it)"
Again I'm not disparaging the program, just trying to point out that it IS a simple procedure.
"I'm running XP Pro SP2 on a Dell M60 laptop"
I'm trying this out for the most part with the same OS and a Dell something or other laptop.
" Boy (now we've worked on this) do I know why you say that like that"
 I know one fellow who built a "class compliant" USB device that worked perfectly except on Russian and Polish localized versions of XP. So you still can't sleep all that soundly. 
But the main reason I bring that up is that the MIDI Sport I'm using requires its own drivers. I'll need to find one of those "class compliant" devices and test it out. As far as I can tell we've eliminated everything else.
"I'll PM you the Sysex dump"
Thanks, and for anyone keeping score that also works fine here.
I'm concerned about what might be going on in those MS black boxes. They might be doing some direct memory access, for example, that makes assumptions about the buffers. But I'll need to get some gear here that doesn't work in order to make any progress on tracking that down.
It would be useful if you could also let me see a failed dump and note the size and number of the buffers if that is at all possible.
Thanks for your report and your willingness to help with the diagnosis.
|
|
IP Logged |
|
|
|
jwpgroup
Member
MIDI-OX Rules!
Gender:
Posts: 12
|
|
Re: Sysex bug
« Reply #15 on: Dec 16th, 2006, 3:30pm » |
Quote Modify
|
Hi Jerry
"So you still can't sleep all that soundly."
Don't! You can imagine how I felt when this 'bug' surfaced.
We delivered what we considered to be final code over a month ago and this came to light only when the final stages of 'in the field reprogramming' where being tested (just before product launch). The code runs on a flash based microcontroller, so it can be field 'updated' to a later version over sysex (well assuming it contains 100% what you send of course!). The subtle changes that affect the MIDI-OX transfer (but not Sysex97) sure had us foxed for a while
"the MIDI Sport I'm using requires its own drivers"
Funny you should say that. We had two (outwardly identical) 'T' boxes here when we where in the early stages of coding (when we looked at several manufacturers devices at USB level). One was 'compliant' whilst the other needed drivers, so you need to be sure you get a later version of whatever you buy. I vaguely remember looking at another M-audio interface (which also wasn't 'compliant'), so I didn't analyse it any further. Maybe that is what you have.
At one point I had about half a dozen units from different manufacturers here so I could have given it a much fuller test (although three failures do look pretty conclusive TBH).
"I'll need to get some gear here that doesn't work in order to make any progress on tracking that down"
The kit I've definitly got it to fail with is... (based on a web search and using American sites in view of your locale)....
http://www.pgmusic.com/th_tbox.htm
This box seems faster than the M-audio unit and (assuming you use 240 byte buffers (to avoid data loss) will pass the whole file with just the data corruption with 0 delays. I'd get this if you want a simple clean box (unbiased view!)
http://www.zzounds.com/item--MDOUNO
Is the M-audio device I have. Note you don't need to load any drivers on XP (although a disk is supplied for W95/W98/2000).
I've found the M-audio box does drop data when the errors happen, but it maybe that this could be fixed by increasing the delays in MIDI-OX (I knew I'd found the problem at that point so was less bothered about getting a file of the correct length!).
Returning to the 'issue'...
The 'compliant' thing is a big thing here. People don't want the faff of having to load drivers - plug n play USB on OSX or XP is the order of the day. If only everyone knew how flawed and patched this whole USB area is (well in XP anyway)! they might take a different view
"I'm concerned about what might be going on in those MS black boxes. They might be doing some direct memory access, for example, that makes assumptions about the buffers"
I think there must be something like that happening and after what we've just been through to get this stuff to work properly I share your concern.
"It would be useful if you could also let me see a failed dump and note the size and number of the buffers if that is at all possible."
I've PM'd you another file. I just created this, so I can make it happen repeatedly when I want to!
This was Tx/Rx'd with 240 byte buffers and zero delay. You'll see 357 bytes that don't match and the same repeating corruption. Any other buffer length leads to data loss at the point of corruption.
Don't forget that Sysex97 always works (might be some clue). You'll have to ignore the revolting music clip when you run it of course
One (unrelated) thing I should mention is I've had several lock-ups on the second version of MIDI-OX when I compared a received file to one loaded (to check for diffrences). If you see that behaviour you might want to clear the moths from that code area as well
"Thanks for your report and your willingness to help with the diagnosis."
Hey thanks for looking at it. The customer wants to recommend MIDI-OX for in-the-field sysex code changes, so I have to admit to a certain degree of 'need' in my request for you to find the bug. It will help us all in the long run.
Happy hunting (& let me know if I can do anything more).
All the best
John
|
|
IP Logged |
|
|
|
JerryJorgenrud
New Member
MIDI-OX Rules!
Posts: 0
|
|
Re: Sysex bug
« Reply #16 on: Dec 16th, 2006, 7:13pm » |
Quote Modify
|
Thanks for the kit info. It looks like a late-model T box for me.
" One (unrelated) thing"
I'll keep an eye out. I hate to keep throwing MS under the bus but we used their rich edit control and have lived to regret it. It works well with the moderately sized data you normally run into but anomalies develop at higher loads. Fortunately we've gotten away with it so far because most people's requirements are light enough to not be an issue. But I wish it worked better and at times it's embarassing to see the control struggling. On the other hand I never wanted to write a full-featured edit control from scratch and still don't.
"It will help us all in the long run"
We live to serve has always been our motto here 
|
|
IP Logged |
|
|
|
Jamie OConnell
Administrator
Gender:
Posts: 2027
|
|
Re: Sysex bug
« Reply #17 on: Dec 17th, 2006, 7:30pm » |
Quote Modify
|
FWIW, I can't cause a failure with the new MIDI Yoke at all, doing transfers, even when specifying 0 delays in MIDI-OX.
I created a SysEx message from the excerpt above and tagged on an F7 to the end (248 Bytes) and then copied it 7 times.
What sort of delays were you using in MIDI-OX?
Configuration:
MIDI-OX (1)->MIDI Yoke 2->MIDI-OX (2)
[SysEx|Receive Manual Dump...]
Settings
(For Both MIDI-OX instances)
Delay [ 0 ] Milliseconds between buffers
[x] Delay after F7 [ 0 ] Milliseconds
Input Buffers: [ 256 ] Bytes x [ 16 ] Buffers
Output Buffers: [ 256 ] Bytes x [ 16 ] Buffers
I also cannot get the EDIROL UM-880 (USB-MIDI) to fail:
MIDI-OX (1)->[2:UM-880]->cable->[UM-880:1]->MIDI-OX (2)
Settings
(For Both MIDI-OX instances)
Delay [ 0 ] Milliseconds between buffers
[x] Delay after F7 [ 0 ] Milliseconds
Input Buffers: [ 256 ] Bytes x [ 16 ] Buffers
Output Buffers: [ 256 ] Bytes x [ 16 ] Buffers
I then copied the message 8 more times (17856 bytes) and sent that. At this point I needed to insert a 10 millisceond delay between buffers to avoid a WINMM "driver busy" message: "[x] Delay after F7 [ 10 ] Milliseconds".
Regardless, I always received exactly the same data as sent (Copying and Pasting the original MIDI message and using the Compare Windows function).
It's possible that all of the USB devices you tested use the same driver code (and or circuits), so it wouldn't be that surprising if several of them exhibited that same failure. It remains to be explained why this SysEx97 program doesn't fail? I wonder what sort of delays or processing overhead it might insert to allow these USB devices to function correctly.
|
| « Last Edit: Dec 17th, 2006, 7:33pm by Jamie OConnell » |
IP Logged |
--Jamie
Music is its own reward.
|
|
|
jwpgroup
Member
MIDI-OX Rules!
Gender:
Posts: 12
|
|
Re: Sysex bug
« Reply #18 on: Dec 17th, 2006, 11:30pm » |
Quote Modify
|
Hi Jamie
"What sort of delays were you using in MIDI-OX?"
With two of the three tested units (ours & the Tahong 'T' box) no delay is necessary (as long as you use a buffer length of 240 bytes). With these two items you get the entire file but with three byte (100% repeatable) corruptions every 240 bytes.
Different buffer lengths cause data loss when the corruption occurs, irrespective of delay setting.
The third test unit (the M-audio unit), finds it hard to keep up and a delay is necessary.
"I also cannot get the EDIROL UM-880 (USB-MIDI) to fail"
Correct me if I'm wrong , but this unit requires a specific driver written by Roland, so I'm not surprised it works. Jerry & I currently think this is a Microsoft 'class compliant' - MIDI-OX interaction issue, so you need a product that doesn't need drivers but depends on the OS.
"It's possible that all of the USB devices you tested use the same driver code (and or circuits)"
That's definetly not the case in the outside world (i.e. leaving aside the OS). This whole 'bug' came to light because we where testing USB 'class compliant' routines which I (and my partner in crime Dave) had personally written, running on a completly different single chip micro to that used in either of the other units. No doubt about that.
Sadly (from a MIDI-OX point of view) this problem has now shown itself on three different micros from three different manufacturers (one of which I'm involved with as a programmer/developer). IMHO, It's fairly heavy proof of a 'bug' when one also considers that there is no issue at all when the code is transmitted with Sysex97.
I think all this points to an interaction 'issue' between MIDI-OX & the 'budled' MS 'class' drivers.
"It remains to be explained why this SysEx97 program doesn't fail?"
That is indeed something to wonder about, but it doesn't.
" I wonder what sort of delays or processing overhead it might insert to allow these USB devices to function correctly."
I've pretty much tried everything I can to get MIDI-OX to work with zero success I'm afraid. It's a really weird bug for sure. It's not delay related - the corruption is always the same and Sysex97 is way faster than MIDI-OX.
Just so you know, I've PMd Jerry the code examples - let me know if you'd like those as well.
Thanks for looking Jamie. I really appreciate how responsive you guys are being over this.
All the best
John
|
|
IP Logged |
|
|
|
Jamie OConnell
Administrator
Gender:
Posts: 2027
|
|
Re: Sysex bug
« Reply #19 on: Dec 19th, 2006, 1:00pm » |
Quote Modify
|
on Dec 17th, 2006, 11:30pm, jwpgroup wrote:
Jerry & I currently think this is a Microsoft 'class compliant' - MIDI-OX interaction issue, so you need a product that doesn't need drivers but depends on the OS.
"It's possible that all of the USB devices you tested use the same driver code (and or circuits)"
That's definetly not the case in the outside world (i.e. leaving aside the OS). This whole 'bug' came to light because we where testing USB 'class compliant' routines which I (and my partner in crime Dave) had personally written, running on a completly different single chip micro to that used in either of the other units. No doubt about that.
|
|
It sounds like you are saying that these devices don't need drivers? But that can't be: it is impossible. All MIDI devices need to use drivers. It is just a question of whether the driver is written completely by the manufacturer (UM-880), partly by the manufacturer and partly by MS (using the boiler plate 'port class') or completely by MS (still using 'port class'). I still suspect that all devices that have this problem are using the class boiler plate driver code.
MIDI-OX does nothing unusual with the buffers it ships. It simply uses midOutLongMessage() to hand the buffers to an open driver (marshalled via MMSYSTEM and WINMM). Proof of what comes out of MIDI-OX can be seen by the MIDI Yoke loopback tests described above. Logic dictates that there must be some sort of problem with the MS 'port class' SysEx implementation.
So that I can attempt to reproduce your results could you specify your MIDI-OX SysEx settings as I have done in the previous message?
|
|
IP Logged |
--Jamie
Music is its own reward.
|
|
|
|
User Forum ::
Powered by YaBB
Home 
Help 
Search 
Members 
Login 
Register
Reply
Notify of replies
Send Topic
Print