Dynamics CRM 365 – On Prem – Invalid Trace Directory

Looks like another piece of CRM team awesomeness.

The Tracing directory should be:

C:\Program Files\Microsoft Dynamics CRM\Trace

 

But some update somewhere changes it to:

c:\crmdrop\logs

 

That’s not very helpful.

I initially tried to change the trace directory back to the right place using CRM PowerShell, but that failed with authentication errors (that i have also posted on here – http://paulobrien.co.nz/2018/03/07/get-crmsetting-powershell-the-caller-was-not-authenticated-by-the-service-the-request-for-security-token-could-not-be-satisfied-because-authentication-failed/).

 

This is the guide i tried using the powershell method – makes sense, if powershell crm wasn’t broken as well.

How to fix ‘Invalid Trace Directory’ errors

 

So ended up changing in the CRM Database and registry:

 

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM]
"TraceDirectory"="C:\\Program Files\\Microsoft Dynamics CRM\\Trace"
"TraceEnabled"=dword:00000001

 

And in the MSCRM_CONFIG database:

SELECT NVarCharColumn
  FROM [MSCRM_CONFIG].[dbo].[ServerSettingsProperties]
  where ColumnName = 'TraceDirectory'

  update [MSCRM_CONFIG].[dbo].[ServerSettingsProperties]
  set NVarCharColumn = 'C:\Program Files\Microsoft Dynamics CRM\Trace'
  where ColumnName = 'TraceDirectory'

 

 

Get-CrmSetting powershell – The caller was not authenticated by the service / The request for security token could not be satisfied because authentication failed.

Had a need to run CRM powershell on an On-premise CRM Server.

I’ve had this same issue before where any powershell command you run against the Microsoft.Crm.PowerShell provider fails with authentication errors.

In this example, i was trying to run this and it barfed on Get-CrmSetting ….

Add-PSSnapin Microsoft.Crm.PowerShell

$ClaimsSettings = Get-CrmSetting -SettingType OAuthClaimsSettings

$ClaimsSettings.Enabled = $true

Set-CrmSetting -Setting $ClaimsSettings

 

Turned out to fix it, i needed to add a registry key.  After i did this it worked straight away (no need to reboot or reopen the PS window).

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa]
"DisableLoopbackCheck"=dword:00000001

 

Method 2 in this article:

https://support.microsoft.com/en-us/help/896861/you-receive-error-401-1-when-you-browse-a-web-site-that-uses-integrate

ESP32 programming with Arduino on Windows

Today, I had an ESP32 development board arrive.  I have been eagerly awaiting this new ESP32 chip as from what I can understand, it has a lot more I/O pins (which you run out of pretty quickly on a ESP8266 dev board like a Wemos) with some more dedicated rs232 interfaces, and the big thing I have been waiting for – 2 cores!  This means in theory you can design your IoT code so one core can handle a web server and communications requests while the other core does time consuming work like blasting RF433 signals and the like – well, so I believe anyway..

So I ordered this chip – https://www.aliexpress.com/item/New-Wireless-module-NodeMcu-Lua-WIFI-Internet-of-Things-development-board-based-ESP8266-with-pcb-Antenna/32266249427.html?spm=2114.13010608.0.0.jnMoB6

Was around $20 delivered to me here in NZ and took just over a month to arrive.  Package turned up all OK.

So I started looking around to see how to setup my Arduino IDE to work with it, but i found a couple of guides that didn’t work so well for me and I had to play around to get it all to work.

Here are the steps I followed.

Make sure i am running Arduino IDE 1.6.12 – maybe if you have an older version, you want to upgrade.

 

Install Python – https://www.python.org/downloads/release/python-2712/ – I installed the 64 bit Windows one.
When the installer runs, all the options are selected except the bottom – Add to Windows Path option – tick this option too as it makes some of the next steps easier having Python in your system path

 

Install a tool called pyserial – https://pypi.python.org/pypi/pyserial
Download the pyserial-3.3.tar.gz file.  I used WinRAR to unzip it in to a folder off my Downloads folder.

 

Open a CMD prompt as Administrator.  Change directory (and drive) to the folder you extracted pyserial-3.3 to.

 

 

Type python setup.py install

You’ll see a whole load of guff on the screen as things get unpacked and installed.  Check through there is no errors.
Keep the CMD window open as we need it again later

.

 

Find your Arduino IDE folder – mine was at C:\Program Files (x86)\Arduino.
Go to the hardware folder, then create a new folder called espresif, then go in to that folder and create another new folder called esp32.

 

Download the Arduino-ESP32 code from https://github.com/espressif/arduino-esp32/archive/master.zip

Extract the contents of this download in to the folder you created above.  Should look like below

 

Go back to the CMD window, change directory to the folder you created above off your Arduino IDE folder.
Once your in the directory, type GET and press Enter

 

Install is complete! 

 

Testing time.

 

Open Arduino IDE.

I went to File > Examples and you should now see an ESP32 section.  I opened Wifi > Wifi Scan

 

Go to Tools and then the Boards sub menu.  Select the ESP32 Dev Module.  I checked the port was right too by plugging and unplugging the ESP32 in to my USB board and going into the Port > sub menu to see it was in there, then assigned – in my case COM6

 

I learnt two things with uploading the image to the board.  One, my trusty USB cable I have used on Wemos and other chips and things just wouldn’t work with the chip.  Once you plugged in the ESP32 board you just heard every 20 seconds or so the USB plug in/unplug sound on the PC.  I ended out swapping for a shorter more ‘robust’ looking cable and plugged in and then had no issues.  Second thing is – Hold down the BOOT button on the ESP32 dev board when uploading from Arduino IDE or the upload fails.  I know there is probably a pin to pull low (I think its pin 9 but I haven’t got to that yet) that will put the chip in to ‘flash’ mode.

 

With the BOOT button held down, press the -> Arrow in Arduino IDE to push the sketch to the chip.

This is a successful upload for me below:

 

With the sketch uploaded to the ESP32, I was away!  Open up the Serial Window and my simple WiFi Scan sketch was running and could see WLANs around me.

 

Now on to figure out what this chips can do.. I know the clever awesome people working on the Arduino stuff for the ESP32 are adding stuff all the time with more and more of the stuff we’ve come to expect on Arduino and ESP8266 chips is ported over.  I see OTA Updates and mDNS have recently been added.

I’m very keen to see a webserver running on one core/task/thread while something else can be consuming the loop () { } and holding it up if it needs to without affecting the performance of the webserver thread…  I’ve tried different ESP8266 web servers, async and interrupt tricks and queuing etc., but if your chip has to go off and do a long time operation, the web server doesn’t respond like if it was only being a web server.  When you’re working with some other IoT Gateway or bridge that is not very forgiving in its timeouts, this can make or break a design.
Or even better yet, the core threaded so you can just call tasks and let the chip assign them to the cores… this will allow for far more complex solutions without having to connect multiple micro controllers together and then talk on a bus like i2c between the chips.

The other big plus I see with these ESP32 chips from what I read is that all the pins are assignable in software, so you can make any logical pin physically route to another pin and it has more than one hardware serial TX/RX so you get hardware performance serial ports in case the ‘thing’ you are talking to is not so easily done with software serial without messing around with the standard development serial connection on RX/TX that is generally wired to the USB RS232 chip on the little dev board.

Anyway, now on to building something worthwhile with this chip and see what I can get it to do.

 

Azure DNS via PowerShell

In my working world, I recently have discovered the relatively new feature in Azure – DNS.

From my early playing round it seems great! Very fast to update and easy to control with scripting.

You will need the Azure PowerShell provider.  Very simple to get.

You basically open up PowerShell and type:

Install-Module AzureRM

And then you wait 5 mins while it gets and installs a whole bunch of stuff..

 

These two links are handy to give you insight in to how to connect and running different commands:

https://docs.microsoft.com/en-us/powershell/azureps-cmdlets-docs/

https://docs.microsoft.com/en-us/azure/dns/dns-operations-dnszones

 

 

Below is a script i wrote that that adds an A record.  It does a few checks and creates the dependent bits if they don’t exist – basically you need a resource group to put the DNS zone in, then you need a DNS zone and then you can create a record.

Couple of things to note from the script – you will want to set your location – my closest is “australiasoutheast” – you will want to find your preferred location to create your resource groups.

Second thing, I found this out by accident but very handy.. you can sign in to AzureRM and then ‘export’ your sign in details to a JSON file where the password etc is encrypted.. then in your script you can just use the Select-AzureRmProfile command and point it at the file, and it takes care of establishing a session with AzureRM.

Export your login details:

#Login - this prompts auth crenetials
Login-AzureRmAccount

#Once you are auth'd you can export your login credentials to an encrypted JSON file
Save-AzureRmProfile -Path “..\azureprofile.json”

 

Here’s the script – you can see it uses the saved credentials above:

param (
	[string] $zone,		  #eg: paultest100.com
	[string] $hostname,	  #eg: www.paultest100.com  
	[string] $address,	  #eg: 10.20.30.40
	[int] $TTL	          #eg: 3600
)

#get record from hostname - strip off the zone
$z = $hostname.Replace($zone, '').TrimEnd('.')

#create resource group name
$rgName = "MyDNSResources"

#Login
Select-AzureRmProfile -Path "..\azureprofile.json"
        
#Get Resource Group - Create if it doesn't exist
$rg = Get-AzureRmResourceGroup -Name $rgName
if ($rg -eq $null)
{
    $rg = New-AzureRmResourceGroup -Name $rgName -location "australiasoutheast"
}

#Get DNS Zone - Create if it doesn't exist
$dnsZone = Get-AzureRmDnsZone -Name $zone -ResourceGroupName $rg.ResourceGroupName
if ($dnsZone -eq $null)
{
    $dnsZone = New-AzureRmDnsZone -Name $zone -ResourceGroupName $rg.ResourceGroupName
}

#Get A record - Create if it doesn't exist
$dnsRecord = Get-AzureRmDnsRecordSet -Name $z -RecordType A -ZoneName $dnsZone.Name -ResourceGroupName $rg.ResourceGroupName
if ($dnsRecord -eq $null)
{
    #Create A Record
    New-AzureRmDnsRecordSet -Name $z -RecordType A -ZoneName $dnsZone.Name -ResourceGroupName $rg.ResourceGroupName -Ttl $TTL -DnsRecords (New-AzureRmDnsRecordConfig -IPv4Address $Address) -Overwrite -Force
}

Google Places API in C# .NET

I recently had a requirement to look in to using the Google Places API in some code i was writing.

I found this library here that made the whole thing a lot easier.

https://github.com/azizmb9494/Google-Places 

 

I ended up writing a quick C# application to test the different searches by lon/lat and by text.

Download Places –  a quick and dirty C# application that implements the API.. you’ll need to get a Google API Key to run it.

https://developers.google.com/places/web-service/get-api-key

 

Paste your key in to the app and then you can search by lon/lat, text or a combination of both

The basic guts of the code is below.  Having never had anything to do with Google API’s before, it was pretty simple to get some programmatic searching going on…

Example method using the API above:

 private async void TestGooglePlaces()
        {
            Response results;
            var placeList = new List<Place>();
            string apiKey = "your api key";

            if ((string.IsNullOrEmpty(txtLat.Text) || string.IsNullOrEmpty(txtLon.Text))) //if we dont have a lon/lat .. search txt
            {
                results = await Places.Api.TextSearch(txtQuery.Text, apiKey);
            }
            else
            {
                results = await Places.Api.SearchPlaces(Convert.ToDouble(txtLat.Text), Convert.ToDouble(txtLon.Text), txtQuery.Text, apiKey);
            }

            //add the results to placeList
            foreach (var place in results.Places)
            {
                placeList.Add(place);
            }

            //if there are more than one 'page' of results...
            while (results.Next != null)
            {
                //get the next lot of results
                results = await Places.Api.GetNext(results.Next, apiKey);

                foreach (var place in results.Places)
                {
                    placeList.Add(place);
                }
            }

            foreach (var place in placeList)
            {
                var placeDetails = await Places.Api.GetDetails(place.PlaceId, apiKey);

                //do stuff with your place and placeDetails                
                string name = place.Name;
                string address = placeDetails.Address;
                //......
            }

        }

IoT development using Arduino, ESP8266, C++ and Wemos development boards

I’ve been playing around with Arduino’s and things now for a few years on and off.  Although I’ve dabbled in electronics before, I was completely new to the world of micro-controllers and C++.

Initially, very cool.. you could have a bit of logic behind some buttons, lights and sensors etc without having to be an electronics guru.  And with the thousands of libraries and example of Arudino sketches that do everything from talking to cars via CANBUS to controlling motors and reading sensors, this is a great way to get started in the world of IoT.  Even if you’re knowledge of C++ is VERY limited like when I started, the scripts you can get show you a lot on the way of how things work,.. and you can generally google a question and someone has an answer…

 

A lot of what I wanted to do relies on communicating with other things, so pretty early in to playing around with it all, I wanted to get some WiFi working and get these Arduino boards collecting some data and maybe uploading it to a database over the web.

I discovered the ESP8266 WIFI Chip.

I am really interested in what could be done with these super low cost chips, because of just that – they are a few dollars, not $50+ just for some comms.

I know people are all over Raspberry PI’s, but to me they are just a computer from 10 years ago running Linux.  Sure, they have their places for media players and all sorts of more advanced IoT bits and pieces where all that computing power is needed, but for low cost little things that do a simple job like controlling a relay or measuring a temperature, they seem over kill and are expensive.

When I first started with it all, these ESP8266 WiFi Chips that retail for a few dollars had just come out.  Initially I spent hours wiring these things up to pins on an Arduino board and sending AT commands to the chip to get responses and things.. and when I say hours, I mean HOURS.  No matter what I did though, I always found them a bit flaky, I could get basic things to work but if I left them running overnight or for a few days, the WiFi chip would have locked up or dropped off, or reset..  The alternative was chips like Ti C3300 but they are all a lot more expensive…

 

I kind of gave up on it all for a while as I didn’t think you could ever really make anything serious with them.

I’m glad I did to be honest.  Because, when I came back to it 7-8 months later, the game had changed, significantly…

 

The first and foremost thing that happened is that some clever people got the actual Arduino concept (C++, Arduino IDE to code in, whole stack of cool libraries that do cool stuff) running directly on the ESP8266 chips themselves, so you didn’t have to wire up an Arduino to and ESP8266 over serial (or software serial), you just wrote code and uploaded to the one ESP8266 chip.

The ESP8266 has a much faster processor (80mhz) than the Atmel processors on the Arduino’s + a lot more memory to run your sketches in.

Also, boards like the NodeMCU had come out that were essentially an ESP8266 on a nice little development board with a USB <> RS232 adapter baked in, all the voltages right etc. (as Arduino was 5v and ESP8266 is 3.3v so you had to mess about with level shifters etc..) – just plug in to your USB port and go without having to build a circuit with Arduino, ESP8266 (that was not friendly to mount on a breadboard – some of them anyway), voltage dividers to step voltage down on data pins, blah blah.. etc etc..

 

The second thing, was that no more did you need to use AT commands like your talking to some modem from the 1980’s.  Someone had written some nice ESP8266 libraries you can just include in your sketches and get going with just a few lines of code.  I’d ordered a bunch of NodeMCU boards from Aliexpress but still found that they would do funny things and not be reliable.

Then in my sniffing around the internet, I discovered the smaller Wemos boards that also had an ESP8266 but they were the ones with the nice metal can over the wifi chip to make them more compliant and the whole package is quite a bit smaller.

 

Since working with these modules now, things are solid.  I don’t know if the NodeMCU boards I had just had flaky ESP8266 chips or power supply components.. but finally after messing about with this stuff for a couple of years, I had a reliable setup..

 

So if anyone wants to fast forward the messing about I’ve done and get stuck in to building something small and Wifi capable using the Arduino community and libraries, here is a list of things I have learnt messing about with all the different versions of this stuff.

 

I also found this great tool that gives you a ready to go WiFi Setup system for your widget.. from a phone, you just connect to the Ad-Hoc network your board creates and it automatically brings up a page to join your main WiFi Network – then device reboots and connects to said network.. makes it so easy!

https://github.com/tzapu/WiFiManager

 

This is a good place to start your development.

 

  • Get yourself the latest Arduino IDE
  • Get your self a WeMOS board – $4.00 US!! – $5.76 delivered to your door here in NZ.
  • Maybe get your self a Wemos Shield – they stack on WeMOS without any wiring, there are things like temperature sensors, little OLED screens, relays etc:
  • Relay
  • Screen

Install Arduino IDE

Go in to Board Manager in IDE and search for Wemos – Install the esp8266 board pack that includes setup for the Wemos board

 

From the top menu – Sketch > Include Libraries – Manage Library – go and find and install WiFiManager

Just to test things are working end to end, load the Example Project Blink from the ESP8266 sub menu in the File > Examples menu

 

Plugin your Wemos board.. On Windows it should plug and play the serial driver and appear as a new com port.

From the Tools menu, select the Wemos board and select the COM port your Wemos is on.

Upload the sketch to the WeMos just like you would an Arduino and confirm the light starts blinking!!

One thing to note here, i had a lot of faffing around with sketches intermittently not uploading to the Wemos.. the process just saying failed.
What i figured out is that you are mean to ground Pin D3 on the Wemos to put it in to flash mode.Sometimes it will upload/flash without you having to ground this pin, but once you get a sketch on there and you’re recompiling and uploading, it can get a bit finiky.
So remember to short between D3 and Ground when uploading sketch, and then take the short off to run.

 

If you’ve got to here, great!!

 

Now, lets get some basic WiFi happening and the WiFi Manager which manages setting up the WiFi connection simply without hard coding into sketches.

 

In the Examples, find WiFi Manager (should be there if you installed it earlier from the library).  Maybe restart the IDE if you dont see it.

Load the AutoConnectWithFeedbackLED sketch and upload this to your Wemos.

Once the project is uploaded successfully, power off and on the Wemos.

Give it 20 seconds or so to start and look for Wifi networks.  Then open up your phone (in my case iPhone).  go to your WiFi networks and choose the ESP….. – whatever your chip is called by default.  You will then see the browser on the phone pop and navigate to http://192.168.4.1 – the default IP of the ESP8266.  It will DHCP your phone/device an address and auto-pop up a config page.  How cool is that!  From here you can find your home Wifi network and connect with SSID/Pass.  Once you save, the device reboots and connects to the AP.  If it fails, it connects back to this setup config mode

                        

If you have a console connected to the serial port the Wemos is on (default 115,200 baud rate), you will see the chip start, connect to your network and get an IP Address.

 

From here, the world is your oyster as they say.  You can measure sensors, send results to web services etc etc..

I thought this was a pretty cool place to start off though.. it has taken years of messing around with different aspects of this, but this above process is now the basis for any of my IoT ideas now.

How i’d wish it had been this easy a couple of years ago back in the day of soldering up boards with level shifters and connecting arduino nano’s to ESP8266 chips through level shifters and then sending a whole bunch of AT commands just to get something to happen….hours of messing around can now be sorted in less than 5 minutes once you are all setup.

The little Wemos shields too instantly give you temp sensors, relays, screens and all sorts of other things you can interact with.. like arudino shields but nice and small

 

Hope that helps someone get started in the wonderful world of IoT.

First Time Caller…

Hello World, as they say.

I’m getting all modern and entering the world of blogging…

I’m a 40 year old lad from NZ who likes playing around with gadgets and things and has worked in the IT industry and Cloud Computing space most of my working life…

I’ve also been messing about with some of these new low cost micro controllers and sensors and IoT things over the past couple of years and thought I would share some of my findings that aren’t always so obvious to figure out.. especially if one is new to the whole thing like I was…

Things are changing every day and you can spend hours researching something, and then find someone has already built something that is exactly what you need and you can implement it in 5 minutes….

Anyways, I spend a lot of time online with work and personal stuff finding answers and shortcuts to what I am trying to do and am forever grateful to all the example code and bits and pieces I have learnt from others online over the years that share their knowledge..

Feeling philanthropic in my old age, I hope something I write can help someone else who may be interested in this whole IoT world that is coming… or maybe the odd script or something I write for work that could shortcut someone else having to do the same thing from scratch…