OpenPilot Chevy Bolt Giraffe Updates

The Generation 3 Giraffe design is complete and tested. And great news! It fits under the plastic cover. Very stealthy :).

You can checkout the schematic and board layout – even build you own here: https://easyeda.com/jshuler/chevy-bolt-openpilot-giraffe-rev-3

Working on this project is expensive, and as a result I have created a Patreon page: https://www.patreon.com/boltopenpilot
If you would like to help support the development efforts I would greatly appreciate whatever you can contribute!

Finally if you are interested in purchasing this Giraffe, head over to the Comma.ai Discord community and look for jshuler.

OpenPilot Bolt Port Stable, Giraffe Available

After many months and countless hours the OpenPilot Chevy Bolt EV port is functional and stable for Lateral (steering) control. I have developed a Giraffe board that dramatically simplifies the connection to the EON.

I am very close to being able to sell assembled Giraffes, however the board is Open Source and you can build your own!

The article has been updated:

Validating Alexa Skill Web Requests in c#

Amazon went a little OCD on the security verification for Alexa skill requests to custom https endpoints. During testing it will work fine if you don’t validate, but they check for this as part of the skill submission process.

I have been developing a custom connector for Alexa to work with the Microsoft Bot Framework, and just recently discovered the security requirements. Now I may be reinventing the wheel here a little bit, but I decided to build my own model classes. Maybe I’m a little OCD.

Anyway, to validate the alexa skill requests you must do the following:
1: Validate that the url supplied in a header for a certificate chain is valid
2: Validate the certificate and its chain
3: Use the certificate to verify the digital signature (which is supplied in another header) against the request body
4: Make sure the timestamp in the request body is within 150 seconds of now

Amazon’s instructions with regard to step 3 are a little misleading. They suggest you “decrypt” the digital signature using the public key. From what I can gather, RSA public keys do not decrypt; they only encrypt. Thankfully the digital signature validation process is something that is already implemented in the .NET Framework, so it’s not terribly difficult.

The two “tricky” parts are getting .NET to read certificates from a PEM container, and checking the signing certificate’s SAN list.

So hopefully this will be helpful to somebody.

First we have a helper class for parsing the PEM and a couple other little things

using System.Net.Http;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Web;

namespace AlexaCustomChannel
{
    public static class PemHelper
    {
        static string CertHeader = "-----BEGIN CERTIFICATE-----";
        static string CertFooter = "-----END CERTIFICATE-----";

        static HttpClient _client = new HttpClient();


        public static IEnumerable<string> ParseSujectAlternativeNames(X509Certificate2 cert)
        {
            Regex sanRex = new Regex(@"^DNS Name=(.*)", RegexOptions.Compiled | RegexOptions.CultureInvariant);

            var sanList = from X509Extension ext in cert.Extensions
                          where ext.Oid.FriendlyName.Equals("Subject Alternative Name", StringComparison.Ordinal)
                          let data = new AsnEncodedData(ext.Oid, ext.RawData)
                          let text = data.Format(true)
                          from line in text.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries)
                          let match = sanRex.Match(line)
                          where match.Success && match.Groups.Count > 0 && !string.IsNullOrEmpty(match.Groups[1].Value)
                          select match.Groups[1].Value;

            return sanList;
        }


        public static bool ValidateCertificateChain(X509Certificate2 certificate, IEnumerable<X509Certificate2> chain)
        {
            using (var verifier = new X509Chain())
            {
                verifier.ChainPolicy.ExtraStore.AddRange(chain.ToArray());
                var result = verifier.Build(certificate);
                return result;
            }
        }

        public static X509Certificate2 ParseCertificate(string base64CertificateText)
        {
            var bytes = Convert.FromBase64String(base64CertificateText);
            X509Certificate2 cert = new X509Certificate2(bytes);
            return cert;
        }

        public static async Task<X509Certificate2[]> DownloadPemCertificatesAsync(string pemUri)
        {
            var pemText = await _client.GetStringAsync(pemUri);
            if (string.IsNullOrEmpty(pemText)) return null;
            return ReadPemCertificates(pemText);
        }


        public static X509Certificate2[] ReadPemCertificates(string pemString)
        {
            var lines = pemString.Split(new char[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries);
            List<string> certList = new List<string>();
            StringBuilder grouper = null;
            for (int i = 0; i < lines.Length; i++)
            {
                var curLine = lines[i];
                if (curLine.Equals(CertHeader, StringComparison.Ordinal))
                {
                    grouper = new StringBuilder();
                }
                else if (curLine.Equals(CertFooter, StringComparison.Ordinal))
                {
                    certList.Add(grouper.ToString());
                    grouper = null;
                }
                else
                {
                    if (grouper != null)
                    {
                        grouper.Append(curLine);
                    }
                }
            }

            List<X509Certificate2> collection = new List<X509Certificate2>();

            foreach (var certText in certList)
            {
                var cert = ParseCertificate(certText);
                collection.Add(cert);
            }

            return collection.ToArray();
        }
    }
}

You will need to change the signature of your controller to accept something that gives you access to the raw request body – such as an HttpRequestMessage.

Then you can call the following method with your request to validate per Amazon’s requirements
(Note the AlexaRequestBody is my custom request model. You just need to get the timestamp from the request)

        static Dictionary<string, X509Certificate2> _validatedCertificateChains = new Dictionary<string, X509Certificate2>();
//...
        async Task ValidateRequestSecurity(HttpRequestMessage httpRequest, byte[] requestBytes, AlexaRequestBody requestBody)
        {
            if (requestBody == null || requestBody.Request == null || requestBody.Request.Timestamp == null)
            {
                throw new InvalidOperationException("Alexa Request Invalid: Request Timestamp Missing");
            }

            var ts = requestBody.Request.Timestamp.Value;
            var tsDiff = (DateTimeOffset.UtcNow - ts).TotalSeconds;

            if (System.Math.Abs(tsDiff) >= 150)
            {
                throw new InvalidOperationException("Alexa Request Invalid: Request Timestamp outside valid range");
            }

            httpRequest.Headers.TryGetValues("SignatureCertChainUrl", out var certUrls);
            httpRequest.Headers.TryGetValues("Signature", out var signatures);

            var certChainUrl = certUrls.FirstOrDefault();
            var signature = signatures.FirstOrDefault();

            if (string.IsNullOrEmpty(certChainUrl))
            {
                throw new InvalidOperationException("Alexa Request Invalid: missing SignatureCertChainUrl header");
            }

            if (string.IsNullOrEmpty(signature))
            {
                throw new InvalidOperationException("Alexa Request Invalid: missing Signature header");
            }

            var uri = new Uri(certChainUrl);

            if (uri.Scheme.ToLower() != "https")
            {
                throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl bad scheme");
            }

            if (uri.Port != 443)
            {
                throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl bad port");
            }

            if (uri.Host.ToLower() != "s3.amazonaws.com")
            {
                throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl bad host");
            }

            if (!uri.AbsolutePath.StartsWith("/echo.api/"))
            {
                throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl bad path");
            }

            X509Certificate2 signingCertificate = null;

            if (!_validatedCertificateChains.ContainsKey(uri.ToString()))
            {
                System.Diagnostics.Trace.WriteLine("Validating cert URL: " + certChainUrl);

                var certList = await PemHelper.DownloadPemCertificatesAsync(uri.ToString());

                if (certList == null || certList.Length < 2)
                {
                    throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl download failed or too few certificates");
                }

                var primaryCert = certList[0];

                var subjectAlternativeNameList = PemHelper.ParseSujectAlternativeNames(primaryCert);

                if (!subjectAlternativeNameList.Contains("echo-api.amazon.com"))
                {
                    throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl certificate missing echo-api.amazon.com from Subject Alternative Names");
                }

                List<X509Certificate2> chainCerts = new List<X509Certificate2>();

                for (int i = 1; i < certList.Length; i++)
                {
                    chainCerts.Add(certList[i]);
                }

                if (!PemHelper.ValidateCertificateChain(primaryCert, chainCerts))
                {
                    throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl certificate chain validation failed");
                }


                signingCertificate = primaryCert;

                lock (_validatedCertificateChains)
                {
                    if (!_validatedCertificateChains.ContainsKey(uri.ToString()))
                    {
                        System.Diagnostics.Trace.WriteLine("Adding validated cert url: " + uri.ToString());
                        _validatedCertificateChains[uri.ToString()] = primaryCert;
                    }
                    else
                    {
                        System.Diagnostics.Trace.WriteLine("Race condition hit while adding validated cert url: " + uri.ToString());
                    }
                }
            }
            else
            {
                signingCertificate = _validatedCertificateChains[uri.ToString()];
            }

            if (signingCertificate == null)
            {
                throw new InvalidOperationException("Alexa Request Invalid: SignatureCertChainUrl certificate generic failure");
            }


            var signatureBytes = Convert.FromBase64String(signature);

            var thing = signingCertificate.GetRSAPublicKey();
            if (!thing.VerifyData(requestBytes, signatureBytes, System.Security.Cryptography.HashAlgorithmName.SHA1, System.Security.Cryptography.RSASignaturePadding.Pkcs1))
            {
                throw new InvalidOperationException("Alexa Request Invalid: Signature verification failed");
            }
        }

Visual Studio Publish Events

Suppose you want to be able to run a command line tool to make changes to the transformed web.config during the publish / deploy of a web project in visual studio.

You need to be able to trigger your command after the web.config is transformed but before it is deployed. Additionally, you need to know the folder used for packaging the deploy.

Here is how.

Right click your project and unload it. Now you can right-click and select Edit the .csproj.

Down near the bottom of the csproj, you can add the following

  <Target Name="MyPublishTask" BeforeTargets="MSDeployPublish">
    <Exec Command="$(ProjectDir)..\ToolFolder\Tool.exe $(ProjectDir)$(IntermediateOutputPath)Package\PackageTmp\ web.config" />
  </Target>

Do not simply name your target “MSDeployPublish” – this overrides the publish functionality. If you use AfterTargets it will run your tool after the deploy finishes. Using BeforeTargets is just about right – it will transform the web.config and copy all the output files to the Package Temp folder. The other piece you need is the path where the package is built – $(ProjectDir)$(IntermediateOutputPath)Package\PackageTmp\ will get you there. Your tool can wreak whatever havoc necessary on the files before they are published.

One you are done editing, reload your project. Now when you publish your tool will run.

Error creating Azure IOT Hub

I was trying to create an IOT Hub in the Azure Portal and the UI was displaying errors on the Pricing and scale tier as well as the Subscription selections.

The error details said “Unable to get property ‘value’ of undefined or null reference”

When I attempted to create the IOT Hub using the azure CLI, we got to the real cause of the issue: the subscription was not enabled for “Microsoft.Devices”

To resolve, follow the directions here to register the Microsoft.Devices resource provider on your subscriptions
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-common-deployment-errors#noregisteredproviderfound

(Open the subscription, look for the “Resource Providers” option, search for “devices” – click register)

Once you have done this, you will be able to create an IOT Hub.

Make SQL Server Collation act similar to SQLite

The default SQL Server database collation (“SQL_Latin1_General_CP1_CI_AS”) sorts some Unicode values as equal when they are not. If you have an nvarchar field defined as part of a primary key or unique index, you can run into some surprise duplicate keys.

In particular, I was loading data from a SQLite database into an Azure SQL (SQL Server) database. I had removed all the duplicates as far as SQLite was concerned, but there were some records that SQL Server complained as being duplicate. From what I can tell, one record used single byte characters for the word “Final” and the other used double-byte characters.

The solution in this case was to change the collation of the field to one that uses a binary sort.

“Latin1_General_100_BIN” seems to work swimmingly. No more strange collisions.