Monday, December 26, 2016

Project Downloads

Source code and binaries of some of my projects are now available at:
http://kamoly.space/projects/

Current projects available include:
http://kamoly.space/projects/comcopy/
http://kamoly.space/projects/gps/
http://kamoly.space/projects/pebble/all_info_as_text_watch_face/
http://kamoly.space/projects/pebble/rpg/

If you happen to download some of these and find them useful, please leave a comment and let me know!

Thanks!

Sunday, December 11, 2016

Retro Computing: Serial File Transfer v1.0

UPDATE: COMCopy binaries and source are now available at:
http://kamoly.space/projects/comcopy/

Getting large / large numbers of files from a modern computer to my old 486 was easy enough (CD-ROM), but since the 486 doesn't have a CD writer the reverse was very difficult. I wanted a way to simply and directly copy from my old computer to my modern one.

Options were to use DOSBox with InterLnk/InterSrv or one of the other many software products out there, but all the options I was seeing were a pain to use. I just wanted a simple mechanism to copy files; not dig through 5 levels of menus or run 25 year old software in an emulator. Embracing any excuse to fire up my old copy of QBasic 4.5, I got to work on making my own.

It's currently just one way (from the old computer to modern), but I'm very happy with how it turned out. There are literally -no- config checkboxes or menus to get it to work. You can change the default folder that files are copied to if you really want, but even that's totally optional.

You run the server program on the modern computer.


It automatically detects the first available COM port and connects to it. You don't even have to click Connect. (The Port drop down and Connect/Disconnect buttons are available on the minuscule chance you actually have 2 COM ports on your modern computer.)

The copy program for the old computer has no configuration at all. For convenience, you'll want to set the PATH environment variable to the location of the executable (or copy the executable to an existing PATH directory).

The PATH environment variable allows you to run DOS executables from any directory you're currently in. When you type in a program name, DOS looks for that file in your current directory and the directories listed in your PATH variable. It's when it fails to see it in either location that it report "Bad command or file name".

Honestly setting the path isn't required, but it makes it super convenient. I just modified my AUTOEXEC.BAT to include a path to my QB45 directory so anything I compile will be readily accessible.


Typing COMCOPY on its own shows help.


Instructions are pretty simple, type:
COMCOPY <source file> <destination path> where <destination path> includes either COM1 or COM2 and an optional target directory/file.

Let's copy over the source code!


It will connect to the server, copy the file, and report the status as it goes along.

When done it will report a simple checksum so you can confirm the file matches on both sides.




You can even specify folders or a full specific path to copy to. I'll make a backup of my drivers...



The biggest technical shortcoming is the checksum calculation. The file copy does a single checksum calculation at the end, but it is possible for it to report an equal value if the failures aligned just right (extremely unlikely though). Better methods would be to continuously check during the copy so it can automatically retransmit incorrect blocks of data. I've never had a copy error (6 foot serial cable at 19200 baud seems pretty reliable), so I'm not overly concerned about it.

This has been a good version 1.0. Future versions need to support wildcards (*.* for example), directory recursion, and handling of a source absolute path. And obviously copying from the modern computer to the old one. For now it satisfies my needs so I'll likely move on to other projects. I hope it is of use to others!


Full source code of the QBasic program (for the DOS based computer):


DEFINT A-Z

commandArgs$ = COMMAND$ '"test.txt COM1"

IF commandArgs$ = "" THEN
   PRINT
   PRINT "COM Copy"
   PRINT "Copy a local file to a destination via the COM port."
   PRINT
   PRINT "Syntax: COMCOPY <FILENAME.EXT> <PORT>"
   PRINT "        COMCOPY <FILENAME.EXT> <PORT>:FILENAME.EXT"
   PRINT "        COMCOPY <FILENAME.EXT> <PORT>:FOLDER\"
   PRINT "        COMCOPY <FILENAME.EXT> <PORT>:FOLDER\FILENAME.EXT"
   PRINT "<PORT> options: COM1 COM2"
   PRINT
   PRINT "Examples:"
   PRINT "COMCOPY MYDATA.XML COM1"
   PRINT "COMCOPY MYDATA.XML COM2:22NOV16.XML"
   PRINT "COMCOPY MYDATA.XML COM1:DATA\"
   PRINT "COMCOPY MYDATA.XML COM2:DATA\22NOV16.XML"
   END
END IF

' Parse the source path.
commandIndex = 1
sourcePath$ = ""
FOR commandIndex = commandIndex TO LEN(commandArgs$)
   indexChar$ = MID$(commandArgs$, commandIndex, 1)
   IF indexChar$ = " " THEN
      commandIndex = commandIndex + 1
      EXIT FOR
   END IF
   sourcePath$ = sourcePath$ + indexChar$
NEXT commandIndex

' Ensure a file name was given.
IF sourcePath$ = "" THEN
   PRINT "Need source file name!"
   END
END IF

' Parse the destination port.
destinationPort$ = ""
FOR commandIndex = commandIndex TO LEN(commandArgs$)
   indexChar$ = MID$(commandArgs$, commandIndex, 1)
   IF indexChar$ = ":" THEN
      commandIndex = commandIndex + 1
      EXIT FOR
   END IF
   destinationPort$ = destinationPort$ + indexChar$
NEXT commandIndex

IF destinationPort$ <> "COM1" AND destinationPort$ <> "COM2" THEN
   PRINT "Invalid Port:" + destinationPort$
   END
END IF

' Parse the destination path.
destinationPath$ = ""
IF commandIndex <= LEN(commandArgs$) THEN
   destinationPath$ = MID$(commandArgs$, commandIndex, LEN(commandArgs$) - commandIndex + 1)
END IF

IF destinationPath$ = "" THEN
   ' No destination path given, use source path.
   destinationPath$ = sourcePath$
ELSEIF RIGHT$(destinationPath$, 1) = "\" THEN
   ' Destination path ends with \, append source file to the end.
   destinationPath$ = destinationPath$ + sourcePath$
END IF

PRINT
PRINT "COM Copy, ESC to cancel"

' Open file for binary transfer.
OPEN sourcePath$ FOR BINARY AS #2
sourceLength& = LOF(2)

' Ensure the file was found / non-zero.
IF sourceLength& = 0 THEN
   PRINT "Nothing to copy!"
   END
END IF

' Open communications (19200 baud, no parity, 8-bit data,
' 1 stop bit, 256-byte input buffer):
' DS0 - Set DSR timeout in milliseconds to 0, effectively off.
' CS0 - Set CTS timeout in milliseconds to 0, effectively off.
' RS - Disable detection of RTS (request to send).
' QBasic 4.5 max baud rate is 19200
OPEN destinationPort$ + ":19200,N,8,1,DS0,CS0,RS" FOR RANDOM AS #1 LEN = 256

' Clear the incoming buffer.
IF NOT EOF(1) THEN
   ' LOC(1) gives the number of characters waiting:
   ModemInput$ = INPUT$(LOC(1), #1)
END IF

' Send initial byte to establish connection.
PRINT #1, "*";

' Need short delay before sending file to establish connection.
SLEEP 1

' Receive the connection response / clear the incoming buffer.
IF NOT EOF(1) THEN
   ModemInput$ = INPUT$(LOC(1), #1)
   IF ModemInput$ = "" THEN
      PRINT "No response from server!"
      END
   END IF
ELSE
   PRINT "No response from server!"
   END
END IF

' Send the destination path.
PRINT #1, destinationPath$;

' Send delimiter.
PRINT #1, "*";

' Send the length we'll be sending.
PRINT #1, STR$(sourceLength&);

' Send a delimiter.
PRINT #1, "*";

PRINT "Sending file "; sourcePath$; ", "; STR$(sourceLength&); " bytes"
PRINT "==10%==20%==30%==40%==50%==60%==70%==80%==90%=100%"

' Setup reporting increment. One * every 2% sent.
byteStatusIncrement! = sourceLength& / 50
statusMarks = 0
checkSum& = 0

' Transfer the file.
FOR fileIndex& = 1 TO sourceLength& STEP 2

   ' Get the data from the file. (This actually gets 2 bytes.)
   GET 2, , dataByte
  
   ' Sends those 2 bytes to the serial port.
   PUT 1, , dataByte

   ' Calculate a simple checksum based on the data sent.
   checkSum& = checkSum& + dataByte
  
   ' Keep the checksum bounded so we don't exceed a long int.
   IF checkSum& > 32767 THEN
      checkSum& = checkSum& - 32767
   ELSEIF checkSum& < -32768 THEN
      checkSum& = checkSum& + 32768
   END IF

   IF INKEY$ = CHR$(27) THEN
      ' User pressed ESC key.
      PRINT "Transfer aborted."
      'EXIT FOR
      END
   END IF

   ' Show indication for each 2% done.
   statusMarksToShow = fileIndex& / byteStatusIncrement!
   WHILE statusMarks < statusMarksToShow
      PRINT "*";
      statusMarks = statusMarks + 1
   WEND

NEXT fileIndex&

' Show final 100% mark.
WHILE statusMarks < 50
   PRINT "*";
   statusMarks = statusMarks + 1
WEND

' Delay to ensure data finishes being sent.
SLEEP 1

PRINT
PRINT "Transfer Complete, Checksum: " + STR$(checkSum&)

' Close the file.
CLOSE 2

' Close the serial port.
CLOSE 1

END


Full source code of the VB.NET program (for the modern Windows based computer):

Imports System.IO.Ports
Imports System.Text

Public Class FormMain

    ''' <summary>
    ''' The serial port object that will communicate.
    ''' </summary>
    ''' <remarks></remarks>
    Private WithEvents connectedSerialPort As New SerialPort()

    Private Class TransferModel
        Public Enum TransferStateEnum As Integer
            Waiting = 0
            ReceivingFilename = 1
            ReceivingSize = 2
            ReceivingData = 3
        End Enum

        Private state As TransferStateEnum = TransferStateEnum.Waiting
        Private destinationPath As String = ""
        Private fileSizeString As String = ""
        Private fileSize_bytes As Integer = 0
        Private lastUpdate_seconds As Double = 0
        Private data As Byte()
        Private dataIndex As Integer = 0
        'Private checkSum As Integer = 0

        Public statusMessage As String = ""

        ''' <summary>
        ''' Local folder on the server that transfers will be saved to.
        ''' </summary>
        ''' <remarks></remarks>
        Public localFolder As String = "C:\COMCOPY\"

        ''' <summary>
        ''' Return a high resolution system time stamp.
        ''' </summary>
        ''' <returns></returns>
        ''' <remarks></remarks>
        Private Function currentSystemTime() As Double
            Dim elapsedTicks As Long = Date.Now.Ticks
            Return elapsedTicks / 10000000.0
        End Function

        ''' <summary>
        ''' Clear temporary variables used in tranfers as reset the transfer state.
        ''' </summary>
        ''' <remarks></remarks>
        Public Sub clear()
            Try
                Me.state = TransferStateEnum.Waiting
                Me.destinationPath = ""
                Me.fileSizeString = ""
                Me.fileSize_bytes = 0
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Check timeout for transfers that have gone stale.
        ''' </summary>
        ''' <remarks></remarks>
        Public Sub checkTimeout()
            Try
                If Me.state <> TransferStateEnum.Waiting Then
                    If Me.currentSystemTime() > (Me.lastUpdate_seconds + 3) Then
                        ' Timed out!
                        Me.clear()
                    End If
                End If
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Process a byte off the network as appropriate for our current transfer state.
        ''' Return if bytes need to be sent back across the network.
        ''' </summary>
        ''' <param name="newByte"></param>
        ''' <param name="responseBytes"></param>
        ''' <returns></returns>
        ''' <remarks></remarks>
        Public Function processIncomingByte(ByVal newByte As Byte, ByRef responseBytes As Byte()) As Boolean
            Dim responseBytesPending As Boolean = False
            Try
                Me.lastUpdate_seconds = Me.currentSystemTime()

                Select Case Me.state
                    Case TransferStateEnum.Waiting
                        If newByte = 42 Then
                            ' Initial byte to establish connection. Return a confirmation byte.
                            responseBytes = {newByte}
                            responseBytesPending = True
                            Me.state = TransferStateEnum.ReceivingFilename
                        End If
                    Case TransferStateEnum.ReceivingFilename
                        If newByte = 42 Then
                            ' Done with filename.
                            Me.destinationPath = Trim(destinationPath)
                            Me.state = TransferStateEnum.ReceivingSize
                        ElseIf newByte <> 0 Then
                            Me.addDestinationPathCharacter(Convert.ToChar(newByte))
                        End If
                    Case TransferStateEnum.ReceivingSize
                        If newByte = 42 Then
                            ' Done with size.
                            Me.prepareForTransfer()
                            Me.state = TransferStateEnum.ReceivingData
                        Else
                            Me.addFileSizeCharacter(Convert.ToChar(newByte))
                        End If
                    Case TransferStateEnum.ReceivingData
                        Me.addData(newByte)
                        If Me.dataIndex >= Me.fileSize_bytes Then
                            ' Done with transfer.
                            Me.completeTransfer()
                            responseBytesPending = False
                        End If
                End Select
            Catch ex As Exception
            End Try
            Return responseBytesPending
        End Function

        ''' <summary>
        ''' Add the latest incoming byte as a destination path character.
        ''' </summary>
        ''' <param name="newCharacter"></param>
        ''' <remarks></remarks>
        Private Sub addDestinationPathCharacter(ByVal newCharacter As Char)
            Try
                Me.destinationPath += newCharacter
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Add the latest incoming byte as a file size character.
        ''' </summary>
        ''' <param name="newCharacter"></param>
        ''' <remarks></remarks>
        Private Sub addFileSizeCharacter(ByVal newCharacter As Char)
            Try
                Me.fileSizeString += newCharacter
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Call to prepare for transfer. Initialize the file data array and reset the checksum.
        ''' </summary>
        ''' <remarks></remarks>
        Private Sub prepareForTransfer()
            Try
                Me.fileSizeString = Trim(Me.fileSizeString)
                Me.fileSize_bytes = Val(Me.fileSizeString)
                Me.data = New Byte(Me.fileSize_bytes - 1) {}
                Me.dataIndex = 0
                'Me.checkSum = 0
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Add incoming data to the new file data array.
        ''' </summary>
        ''' <param name="newByte"></param>
        ''' <remarks></remarks>
        Private Sub addData(ByVal newByte As Byte)
            Try
                Me.data(Me.dataIndex) = newByte
                Me.dataIndex += 1
            Catch ex As Exception
            End Try
        End Sub

        ''' <summary>
        ''' Call when the transfer has completed. This will save the file to the given path and
        ''' reply a checksum value that can be used to ensure successful transfer.
        ''' </summary>
        ''' <remarks></remarks>
        Private Sub completeTransfer()
            Try
                Dim fileFullPath As String = localFolder + destinationPath
                System.IO.File.WriteAllBytes(fileFullPath, Me.data)

                Dim checkSumValue As Integer = 0
                For checkSumIndex As Integer = 0 To (Me.data.Length - 1) Step 2
                    If checkSumIndex = Me.data.Length - 1 Then
                        ' Last byte.
                        checkSumValue += Me.data(checkSumIndex)
                    Else
                        Dim data1 As Int16 = Me.data(checkSumIndex + 1)
                        Dim data0 As Int16 = Me.data(checkSumIndex)
                        Dim newData As Int16 = ((data1 << 8) Or data0)
                        checkSumValue += newData
                    End If
                    If checkSumValue > 32767 Then
                        checkSumValue -= 32767
                    ElseIf checkSumValue < -32768 Then
                        checkSumValue += 32768
                    End If
                Next

                statusMessage = fileFullPath + ", Checksum: " + checkSumValue.ToString + vbCrLf + statusMessage
                Me.clear()
            Catch ex As Exception
            End Try
        End Sub
    End Class

    Private transferObject As TransferModel

    Public Sub New()

        ' This call is required by the designer.
        InitializeComponent()

        ' Add any initialization after the InitializeComponent() call.
        transferObject = New TransferModel
        txtLocalPath.Text = transferObject.localFolder

    End Sub

    Private Sub FormMain_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Try
            refreshPortsList()

            If ddlCOMPort.SelectedIndex >= 0 Then
                ' Port is found, attempt to connect to it.
                connectSerialPort()
            Else
                ' Update which controls are enabled (so disconnect button is disabled at startup).
                updateConnectedEnableFields()
            End If
        Catch ex As Exception
        End Try
    End Sub

    Private Sub btnConnect_Click(sender As Object, e As EventArgs) Handles btnConnect.Click
        Try
            connectSerialPort()
        Catch ex As Exception
        End Try
    End Sub

    Private Sub btnDisconnect_Click(sender As Object, e As EventArgs) Handles btnDisconnect.Click
        Try
            disconnectSerialPort()
        Catch ex As Exception
        End Try
    End Sub

    Private Sub connectedSerialPort_DataReceived(sender As Object, e As SerialDataReceivedEventArgs) Handles connectedSerialPort.DataReceived
        Try
            ' Received data from the serial port.
            Dim receivedSize As Integer = connectedSerialPort.BytesToRead
            Dim receiveBuffer As Byte() = New Byte(receivedSize - 1) {}
            connectedSerialPort.Read(receiveBuffer, 0, receivedSize)

            ' Check if our last transfer has timed out.
            transferObject.checkTimeout()

            If receiveBuffer.Length > 0 Then
                ' Data received, process each incoming byte in the buffer.
                Dim responseBytes As Byte() = {}
                For bufferLoop As Integer = 0 To (receiveBuffer.Length - 1)
                    If transferObject.processIncomingByte(receiveBuffer(bufferLoop), responseBytes) Then
                        ' Response bytes need to be sent, send now.
                        connectedSerialPort.Write(responseBytes, 0, responseBytes.Length)
                    End If
                Next
            End If

        Catch ex As Exception
        End Try
    End Sub

    Private Sub connectedSerialPort_ErrorReceived(sender As Object, e As SerialErrorReceivedEventArgs) Handles connectedSerialPort.ErrorReceived
        Try
        Catch ex As Exception
        End Try
    End Sub

    Private Sub btnRefreshPorts_Click(sender As Object, e As EventArgs) Handles btnRefreshPorts.Click
        Try
            refreshPortsList()
        Catch ex As Exception
        End Try
    End Sub

    ''' <summary>
    ''' Returns if the serial port is currently connected.
    ''' </summary>
    ''' <value></value>
    ''' <returns></returns>
    ''' <remarks></remarks>
    Private ReadOnly Property isSerialPortConnected
        Get
            Return connectedSerialPort.IsOpen()
        End Get
    End Property

    ''' <summary>
    ''' Update whether or not each UI control should be enabled based on
    ''' the connection state.
    ''' </summary>
    ''' <remarks></remarks>
    Private Sub updateConnectedEnableFields()
        Try
            Dim isConnected As Boolean = isSerialPortConnected
            btnConnect.Enabled = Not isConnected
            btnDisconnect.Enabled = isConnected
            ddlCOMPort.Enabled = Not isConnected
            btnRefreshPorts.Enabled = Not isConnected
        Catch ex As Exception
        End Try
    End Sub

    ''' <summary>
    ''' Connect to the selected serial port.
    ''' </summary>
    ''' <remarks></remarks>
    Private Sub connectSerialPort()
        Try
            If ddlCOMPort.SelectedIndex >= 0 Then
                connectedSerialPort.PortName = ddlCOMPort.SelectedItem
                connectedSerialPort.BaudRate = 19200

                connectedSerialPort.WriteTimeout = -1
                connectedSerialPort.ReadTimeout = -1

                connectedSerialPort.ReadBufferSize = 4096
                connectedSerialPort.WriteBufferSize = 2048

                connectedSerialPort.Parity = IO.Ports.Parity.None
                connectedSerialPort.StopBits = IO.Ports.StopBits.One
                connectedSerialPort.DataBits = 8
                connectedSerialPort.Open()
            End If
            updateConnectedEnableFields()
        Catch ex As Exception
        End Try
    End Sub

    ''' <summary>
    ''' Disconnect the serial port.
    ''' </summary>
    ''' <remarks></remarks>
    Private Sub disconnectSerialPort()
        Try
            connectedSerialPort.Close()
            updateConnectedEnableFields()
        Catch ex As Exception
        End Try
    End Sub

    ''' <summary>
    ''' Detect available serial ports and update the COM port list.
    ''' </summary>
    ''' <remarks></remarks>
    Private Sub refreshPortsList()
        Try
            ' Keep track of the selected port so we can reselect it after fresh.
            Dim currentlySelectedPort As String = ddlCOMPort.SelectedItem

            ' Get the available ports and update the dropdown list.
            Dim availablePorts As Array = IO.Ports.SerialPort.GetPortNames()
            ddlCOMPort.Items.Clear()
            ddlCOMPort.Items.AddRange(availablePorts)

            ' Reselect the previously selected port as a default.
            If currentlySelectedPort IsNot Nothing Then
                If ddlCOMPort.Items.Contains(currentlySelectedPort) Then
                    ddlCOMPort.SelectedItem = currentlySelectedPort
                End If
            End If

            If ddlCOMPort.SelectedIndex = -1 AndAlso ddlCOMPort.Items.Count > 0 Then
                ' No port selected but one is available, default to it.
                ddlCOMPort.SelectedIndex = 0
            End If
        Catch ex As Exception
        End Try
    End Sub

    Private Sub btnLocalPath_Click(sender As Object, e As EventArgs) Handles btnLocalPath.Click
        Try
            Dim openFolderDialog As New FolderBrowserDialog
            openFolderDialog.SelectedPath = txtLocalPath.Text
            openFolderDialog.Description = "Select folder to save incoming transfers to."

            If openFolderDialog.ShowDialog() = Windows.Forms.DialogResult.OK Then
                txtLocalPath.Text = openFolderDialog.SelectedPath + "\"
                transferObject.localFolder = txtLocalPath.Text
            End If
        Catch ex As Exception
        End Try
    End Sub

    Private Sub timerStatus_Tick(sender As Object, e As EventArgs) Handles timerStatus.Tick
        Try
            txtReceivedFiles.Text = transferObject.statusMessage
        Catch ex As Exception
        End Try
    End Sub
End Class

Screenshot of the VB.NET designer:





Copyright (c) 2016 Clinton Kam
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.