{"id":3024,"date":"2017-08-23T06:27:22","date_gmt":"2017-08-23T06:27:22","guid":{"rendered":"http:\/\/bicortex.com\/?p=3024"},"modified":"2017-08-24T08:54:28","modified_gmt":"2017-08-24T08:54:28","slug":"using-aws-polly-and-ibm-watson-text-to-speech-and-tone-analyser-artificial-intelligence-services-to-read-and-analyse-clinical-chat-data-part-2","status":"publish","type":"post","link":"http:\/\/bicortex.com\/bicortex\/using-aws-polly-and-ibm-watson-text-to-speech-and-tone-analyser-artificial-intelligence-services-to-read-and-analyse-clinical-chat-data-part-2\/","title":{"rendered":"Using AWS Polly And IBM Watson Text-To-Speech And Tone Analyser Artificial Intelligence Services To Read and Analyse Clinical Chat Data (Part 2)"},"content":{"rendered":"

Note: Part one to this series can be found HERE<\/a><\/p>\n

In my last blog post I outlined the concept of creating a simple Python GUI application which utilised Amazon Polly Text-To-Speech cloud API. The premise was quite simple \u2013 retrieve chat data stored in SQL Server database and pass it to Polly API to convert it into audible stream using a choice of different male and female voices.<\/p>\n

Whilst this functionality provided a good ‘playground’ to showcase one of the multitude of cloud-enabled machine learning applications, I felt that augmenting text-to-voice feature with some visual clues as a representation of the chat content would provide additional value. This is where I thought pairing text-to-speech with linguistic analysis can make this app even more useful and complete. As of today, all major cloud juggernauts offer a plethora general-purpose ML services but when it comes to linguistic analysis which goes beyond sentiment tagging, IBM has risen to become a major player in this arena. IBM Watson Tone Analyser<\/a> specifically targets understanding emotions and communication style using linguistic analysis to detect emotional, social and language tones in written text. Tones detected within the \u2018General Purpose Endpoint\u2019 include joy, fear, sadness, anger, disgust, analytical, confident, tentative, openness, conscientiousness, extraversion, agreeableness, and emotional range. Typical use cases for this service include analysing emotions and tones in what people write online, like tweets or reviews, predicting whether they are happy, sad, confident as well as monitoring customer service and support conversations, personalised marketing and of course chat bots. The following diagram shows the basic flow of calls to the service.<\/p>\n

\"\"<\/a><\/p>\n

You authenticate to the Tone Analyzer API by providing the username and password that are provided in the service credentials for the service instance that you want to use. The API uses HTTP basic authentication. The request includes several parameters and their respective value options and the simplest way to kick some tires (after completing the sigh-up process) is to use curl tool command and some sample text for analysis e.g.<\/p>\n

\r\ncurl -v -u "username":"password" -H "Content-Type: text\/plain" -d "I feel very happy today!"\r\n"https:\/\/gateway.watsonplatform.net\/tone-analyzer\/api\/v3\/tone?version=2016-05-19"\r\n<\/pre>\n
\r\n{\r\n   "document_tone": {\r\n      "tone_categories": [\r\n         {\r\n            "tones": [\r\n               {\r\n                  "score": 0.013453,\r\n                  "tone_id": "anger",\r\n                  "tone_name": "Anger"\r\n               },\r\n               {\r\n                  "score": 0.017433,\r\n                  "tone_id": "disgust",\r\n                  "tone_name": "Disgust"\r\n               },\r\n               {\r\n                  "score": 0.039234,\r\n                  "tone_id": "fear",\r\n                  "tone_name": "Fear"\r\n               },\r\n               {\r\n                  "score": 0.857981,\r\n                  "tone_id": "joy",\r\n                  "tone_name": "Joy"\r\n               },\r\n               {\r\n                  "score": 0.062022,\r\n                  "tone_id": "sadness",\r\n                  "tone_name": "Sadness"\r\n               }\r\n            ],\r\n            "category_id": "emotion_tone",\r\n            "category_name": "Emotion Tone"\r\n         },\r\n         {\r\n            "tones": [\r\n               {\r\n                  "score": 0,\r\n                  "tone_id": "analytical",\r\n                  "tone_name": "Analytical"\r\n               },\r\n               {\r\n                  "score": "0.849827",\r\n                  "tone_id": "confident",\r\n                  "tone_name": "Confident"\r\n               },\r\n               {\r\n                  "score": 0,\r\n                  "tone_id": "tentative",\r\n                  "tone_name": "Tentative"\r\n               }\r\n            ],\r\n            "category_id": "language_tone",\r\n            "category_name": "Language Tone"\r\n         },\r\n         {\r\n            "tones": [\r\n               {\r\n                  "score": 0.016275,\r\n                  "tone_id": "openness_big5",\r\n                  "tone_name": "Openness"\r\n               },\r\n               {\r\n                  "score": 0.262399,\r\n                  "tone_id": "conscientiousness_big5",\r\n                  "tone_name": "Conscientiousness"\r\n               },\r\n               {\r\n                  "score": 0.435574,\r\n                  "tone_id": "extraversion_big5",\r\n                  "tone_name": "Extraversion"\r\n               },\r\n               {\r\n                  "score": 0.679046,\r\n                  "tone_id": "agreeableness_big5",\r\n                  "tone_name": "Agreeableness"\r\n               },\r\n               {\r\n                  "score": 0.092516,\r\n                  "tone_id": "emotional_range_big5",\r\n                  "tone_name": "Emotional Range"\r\n               }\r\n            ],\r\n            "category_id": "social_tone",\r\n            "category_name": "Social Tone"\r\n         }\r\n      ]\r\n   }\r\n}\r\n<\/pre>\n
The service returns JSON structure which can be further unpacked and analysed\/visualised. Using their SDK and a little bit of Python we can create a little script that will pass the desired text to the Tone Analyser API and return a matplotlib graph chart visualising each tone value within its respective category. Below is a simple visualisation of a paragraph containing text with linguistically-negative sentiment and the Python code generating it.<\/p>\n
\r\nimport matplotlib.pyplot as plt\r\nimport numpy as np\r\nimport matplotlib as mpl\r\nimport watson_developer_cloud as wdc\r\n\r\ntone_analyzer = wdc.ToneAnalyzerV3(\r\n  version='2016-05-19',\r\n  username='username',\r\n  password='password',\r\n  x_watson_learning_opt_out=True\r\n)\r\n\r\nmessage = 'Hi Team, I know the times are difficult! \\\r\nOur sales have been disappointing for the \\\r\npast three quarters for our data analytics \\\r\nproduct suite. We have a competitive data \\\r\nanalytics product suite in the industry. \\\r\nBut we need to do our job selling it!'\r\n\r\ntone=tone_analyzer.tone(message, sentences=False, content_type='text\/plain')\r\n\r\n#assign each tone name and value to its respective category \r\nemotion_tone={}\r\nlanguage_tone={}\r\nsocial_tone={}\r\n\r\nfor cat in tone['document_tone']['tone_categories']:\r\n    print('Category:', cat['category_name'])\r\n    if cat['category_name'] == 'Emotion Tone':\r\n        for tone in cat['tones']:\r\n            print('-', tone['tone_name'], tone['score'])\r\n            emotion_tone.update({tone['tone_name']:tone['score']})     \r\n    if cat['category_name'] == 'Social Tone':\r\n        for tone in cat['tones']:\r\n            print('-', tone['tone_name'], tone['score'])\r\n            social_tone.update({tone['tone_name']:tone['score']}) \r\n    if cat['category_name'] == 'Language Tone':\r\n        for tone in cat['tones']:\r\n            print('-', tone['tone_name'], tone['score'])\r\n            language_tone.update({tone['tone_name']:tone['score']})             \r\n\r\n\r\n#find largest value in all tones to adjust the x scale accordingly\r\nmax_tone_value = {**emotion_tone, **language_tone, **social_tone}\r\nif max(max_tone_value.values()) > 0.9:\r\n    max_tone_value = 1\r\nelse:\r\n    max_tone_value = max(max_tone_value.values())+0.1\r\n\r\n\r\n#plot all tones by category\r\nfig = plt.figure(figsize=(7,7))\r\nmpl.style.use('seaborn')\r\nfig.suptitle('Tones by Intensity, scale range: 0(min) - 1(max)', fontsize=14, fontweight='bold')\r\n\r\nx1=fig.add_subplot(311)\r\ny_pos = np.arange(len(emotion_tone.keys()))\r\nplt.barh(y_pos, emotion_tone.values(), align='center', alpha=0.6, color='limegreen')\r\nplt.yticks(y_pos, emotion_tone.keys())\r\nplt.title('Emotion Tone', fontsize=12)\r\nx1.set_xlim([0, max_tone_value])\r\n\r\nx2=fig.add_subplot(312)\r\ny_pos = np.arange(len(social_tone.keys()))\r\nplt.barh(y_pos, social_tone.values(), align='center', alpha=0.6,color='red')\r\nplt.yticks(y_pos, social_tone.keys())\r\nplt.title('Social Tone',fontsize=12)\r\nx2.set_xlim([0, max_tone_value])\r\n\r\nx3=fig.add_subplot(313)\r\ny_pos = np.arange(len(language_tone.keys()))\r\nplt.barh(y_pos, language_tone.values(), height = 0.4, align='center', alpha=0.6, color='deepskyblue')\r\nplt.yticks(y_pos, language_tone.keys())\r\nplt.title('Language Tone',fontsize=12)\r\nx3.set_xlim([0, max_tone_value])\r\n\r\nplt.tight_layout(pad=0.9, w_pad=0.5, h_pad=1.7)\r\nfig.subplots_adjust(top=0.85, left=0.20)\r\nplt.show()\r\n<\/pre>\n
\"\"<\/a><\/p>\n
And finally, the amended Python code for the complete application (including AWS Polly integration from Part 1<\/a>) is as follows:<\/p>\n
\r\nimport sys\r\nimport time\r\nimport io\r\nfrom contextlib import closing\r\nimport multiprocessing\r\nimport pygame\r\nimport numpy as np\r\nimport pyodbc\r\nimport boto3\r\nimport watson_developer_cloud as wdc\r\nimport tkinter as tk\r\nfrom tkinter import scrolledtext, ttk, messagebox\r\nimport matplotlib as mpl\r\nimport matplotlib.pyplot as plt\r\nfrom matplotlib.backends.backend_tkagg import FigureCanvasTkAgg\r\n\r\n\r\nclass ConnectionInfo:\r\n    def __init__(self):\r\n        self.use_win_auth = tk.IntVar()\r\n        self.inst_srv = tk.StringVar()\r\n        self.inst_db = tk.StringVar()\r\n        self.inst_login = tk.StringVar()\r\n        self.inst_passwd = tk.StringVar()\r\n        self.session_id = tk.IntVar()\r\n        self.use_aws_api = tk.IntVar(value=1)\r\n        self.aws_access_key_id = tk.StringVar()\r\n        self.aws_secret_access_key = tk.StringVar()\r\n        self.use_ibm_api = tk.IntVar(value=1)\r\n        self.ibm_username = tk.StringVar()\r\n        self.ibm_passwd = tk.StringVar()\r\n        self.clinician_voice = tk.StringVar()\r\n        self.patient_voice = tk.StringVar()\r\n\r\n        self.ibm_version = '2016-05-19'\r\n        self.ibm_x_watson_learning_opt_out = True\r\n\r\n\r\nclass MsSqlDatabase:\r\n    ODBC_DRIVER = '{ODBC Driver 13 for SQL Server}'\r\n\r\n    def __init__(self, conn_info):\r\n        self.conn_info = conn_info\r\n\r\n    def connect(self):\r\n        connection_string = ('DRIVER={driver};SERVER={server};DATABASE={db};'.format(\r\n            driver=self.ODBC_DRIVER,\r\n            server=self.conn_info.inst_srv.get(),\r\n            db=self.conn_info.inst_db.get()))\r\n        if self.conn_info.use_win_auth.get() == 1:\r\n            connection_string = connection_string + 'Trusted_Connection=yes;'\r\n        else:\r\n            connection_string = connection_string + 'UID={uid};PWD={password};'.format(\r\n                uid=self.conn_info.inst_login.get(),\r\n                password=self.conn_info.inst_passwd.get())\r\n\r\n        try:\r\n            conn = pyodbc.connect(connection_string, timeout=1)\r\n        except pyodbc.Error as err:\r\n            conn = None\r\n        return conn\r\n\r\n    def get_session(self, conn):\r\n        try:\r\n            cursor = conn.cursor()\r\n            cursor.execute(\r\n                """SELECT UPPER(user_role), message_body FROM dbo.test_dialog t\r\n                WHERE t.Session_ID = ? ORDER BY t.ID ASC""", self.conn_info.session_id.get())\r\n            results = cursor.fetchall()\r\n        except pyodbc.Error as err:\r\n            results = None\r\n        return results\r\n\r\n    def get_user_id(self, conn):\r\n        try:\r\n            cursor = conn.cursor()\r\n            cursor.execute(\r\n                """SELECT DISTINCT user_id from dbo.test_dialog t\r\n                WHERE t.session_id = ? AND user_role = 'client'""", self.conn_info.session_id.get())\r\n            results = cursor.fetchall()\r\n        except pyodbc.Error as err:\r\n            results = None\r\n        return results\r\n\r\n    def get_messages(self, conn):\r\n        try:\r\n            cursor = conn.cursor()\r\n            cursor.execute(\r\n                """SELECT t.user_role, t.direction, LTRIM(RTRIM(f.RESULT)) AS message FROM dbo.test_dialog t\r\n                CROSS APPLY dbo.tvf_getConversations (t.message_body, 50, '.') f WHERE t.session_id = ?\r\n                ORDER BY t.id, f.id""", self.conn_info.session_id.get())\r\n            results = cursor.fetchall()\r\n        except pyodbc.Error as err:\r\n            results = None\r\n        return results\r\n\r\n    def get_messages_for_tone_analyse(self, conn):\r\n        try:\r\n            cursor = conn.cursor()\r\n            cursor.execute(\r\n                """DECLARE @message VARCHAR(MAX) \r\n                SELECT @message = COALESCE(@message + ' ', '') + message_body \r\n                FROM dbo.test_dialog t WHERE t.Session_ID = ? AND user_role = 'client' ORDER BY t.ID ASC\r\n                SELECT @message""", self.conn_info.session_id.get())\r\n            results = cursor.fetchall()\r\n            results = [row[0] for row in results]\r\n            try:\r\n                messages = ''.join(results)\r\n            except TypeError:\r\n                messages = None\r\n        except pyodbc.Error as err:\r\n            messages = None\r\n        return messages\r\n\r\n\r\nclass AudioPlayer:\r\n    def __init__(self, credentials, voices):\r\n        self.credentials = credentials\r\n        self.voices = voices\r\n\r\n    def run(self, messages, voices, commands, status):\r\n        status['code'] = 0\r\n        status['message'] = 'OK'\r\n\r\n        try:\r\n            polly_service = boto3.client(\r\n                'polly',\r\n                aws_access_key_id = self.credentials['aws_access_key'],\r\n                aws_secret_access_key = self.credentials['aws_secret_key'],\r\n                region_name = 'eu-west-1')\r\n        except:\r\n            polly_service = None\r\n\r\n        if not polly_service:\r\n            status['code'] = 1\r\n            status['message'] = 'Cannot connect to AWS Polly service. Please check your API credentials are valid.'\r\n            return\r\n\r\n        is_stopped = False\r\n        is_paused = False\r\n        pygame.mixer.init(channels=1, frequency=44100)\r\n        for message in messages:\r\n            print(message)\r\n\r\n            try:\r\n                polly_response = polly_service.synthesize_speech(\r\n                    OutputFormat='ogg_vorbis',\r\n                    Text=message[2],\r\n                    TextType='text',\r\n                    VoiceId=voices[message[0]])\r\n            except:\r\n                polly_response = None\r\n\r\n            if not polly_response:\r\n                status['code'] = 2\r\n                status['message'] = 'Cannot connect to AWS Polly service. Please check your API credentials are valid.'\r\n                break\r\n\r\n            if "AudioStream" in polly_response:\r\n                with closing(polly_response["AudioStream"]) as stream:\r\n                    data = stream.read()\r\n                    filelike = io.BytesIO(data)\r\n                    sound = pygame.mixer.Sound(file=filelike)\r\n                    sound.play()\r\n\r\n                    while pygame.mixer.get_busy() or is_paused:\r\n                        if not commands.empty():\r\n                            command = commands.get()\r\n                            if command == 'STOP':\r\n                                sound.stop()\r\n                                is_stopped = True\r\n                                break\r\n                            if command == 'PAUSE':\r\n                                is_paused = not is_paused\r\n                                if is_paused:\r\n                                    sound.stop()\r\n                                else:\r\n                                    sound.play()\r\n                        time.sleep(0.010)\r\n            if is_stopped:\r\n                break\r\n\r\n\r\nclass AppFrame(object):\r\n    def __init__(self):\r\n        self.root = tk.Tk()\r\n        self.root.title('Polly Text-To-Speech GUI Prototype ver 1.1')\r\n        self.root.resizable(width=False, height=False)\r\n\r\n        self.conn_info = ConnectionInfo()\r\n\r\n        self.menubar = self.create_menubar()\r\n        self.connection_details_frame = ConnDetailsFrame(self.root, self)\r\n        self.session_frame = SessionDetailsFrame(self.root, self)\r\n        self.playback_frame = PlaybackDetailsFrame(self.root, self)\r\n        self.graph_frame = WatsonGraphDetailsFrame(self.root, self)\r\n\r\n    def create_menubar(self):\r\n        menubar = tk.Menu(self.root)\r\n\r\n        title_menu = tk.Menu(menubar, tearoff=0)\r\n        title_menu.add_command(label='API details...', command=self.on_api_details_select)\r\n        title_menu.add_command(label='About...', command=self.on_about_select)\r\n        menubar.add_cascade(label='About', menu=title_menu)\r\n        self.root.config(menu=menubar)\r\n\r\n        return menubar\r\n\r\n    def on_api_details_select(self):\r\n        dialog = APIDetailsDialog(self.root)\r\n        self.root.wait_window(dialog)\r\n\r\n    def on_about_select(self):\r\n        tk.messagebox.showinfo(title="About", message="Polly Text-To-Speech GUI Prototype ver 1.1")\r\n\r\n    def run(self):\r\n        self.root.mainloop()\r\n\r\n\r\nclass ConnDetailsFrame(ttk.LabelFrame):\r\n\r\n    def __init__(self, root, parent):\r\n        super(ConnDetailsFrame, self).__init__(root, text='1. Connection Details')\r\n        super(ConnDetailsFrame, self).grid(\r\n            row=0, column=0, columnspan=3, sticky='W',\r\n            padx=5, pady=5, ipadx=5, ipady=5\r\n        )\r\n\r\n        self.root = root\r\n        self.parent = parent\r\n        self.conn_info = parent.conn_info\r\n\r\n        self.create_notebook()\r\n\r\n    def create_notebook(self):\r\n        self.tab_control = ttk.Notebook(self)\r\n        self.create_frames()\r\n        self.create_labels()\r\n        self.create_entry()\r\n        self.create_checkbuttons()\r\n\r\n    def create_frames(self):\r\n        self.tab_db = ttk.Frame(self.tab_control)\r\n        self.tab_api = ttk.Frame(self.tab_control)\r\n        self.tab_control.add(self.tab_db, text="Database Connection Details ")\r\n        self.tab_control.add(self.tab_api, text="APIs Connection Details ")\r\n        self.tab_control.grid(row=0, column=0, sticky='E', padx=5, pady=5)\r\n\r\n    def create_labels(self):\r\n        ttk.Label(self.tab_db, text="Server\/Instance Name:").grid(row=0, column=0, sticky='E', padx=5, pady=(15, 5))\r\n        ttk.Label(self.tab_db, text="Database Name:").grid(row=1, column=0, sticky='E', padx=5, pady=5)\r\n        ttk.Label(self.tab_db, text="User Name:").grid(column=0, row=3, sticky="E", padx=5, pady=5)\r\n        ttk.Label(self.tab_db, text="Password:").grid(column=0, row=4, sticky="E", padx=5, pady=(5, 10))        \r\n\r\n        ttk.Label(self.tab_api, text="AWS Access Key ID:").grid(column=0, row=1, sticky="E", padx=5, pady=(5, 5))\r\n        ttk.Label(self.tab_api, text="AWS Secret Access Key:").grid(column=0, row=2, sticky="E", padx=5, pady=5)\r\n        ttk.Label(self.tab_api, text="IBM Watson Username:").grid(column=0, row=4, sticky="E", padx=5, pady=(5, 5))\r\n        ttk.Label(self.tab_api, text="IBM Watson Password:").grid(column=0, row=5, sticky="E", padx=5, pady=(5,15))\r\n    \r\n    def create_checkbuttons(self):\r\n        check_use_win_auth = ttk.Checkbutton(self.tab_db, onvalue=1, offvalue=0,\r\n                                             variable=self.conn_info.use_win_auth,\r\n                                             text='Use Windows Authentication',\r\n                                             command=self.on_use_win_auth_change)\r\n        check_use_win_auth.grid(row=2, column=0, sticky='W', padx=15, pady=(15,5))  \r\n        check_use_aws_api = ttk.Checkbutton(self.tab_api, onvalue=1, offvalue=0,\r\n                                            variable=self.conn_info.use_aws_api, text='Use AWS Text-To-Speech API')\r\n        check_use_aws_api.grid(row=0, column=0, sticky='W', padx=15, pady=(15,5)) \r\n        check_use_ibm_api = ttk.Checkbutton(self.tab_api, onvalue=1, offvalue=0,\r\n                                            variable=self.conn_info.use_ibm_api,\r\n                                            text='Use IBM Watson API',\r\n                                            command=self.on_use_ibm_api_change)\r\n        check_use_ibm_api.grid(row=3, column=0, sticky='W', padx=15, pady=(15,5))   \r\n\r\n    def create_entry(self):\r\n        entry_db_server_name = ttk.Entry(self.tab_db, width=60, textvariable=self.conn_info.inst_srv)\r\n        entry_db_server_name.grid(row=0, column=1, sticky='W', padx=10, pady=(15, 5))\r\n        entry_db_name = ttk.Entry(self.tab_db, width=60, textvariable=self.conn_info.inst_db)\r\n        entry_db_name.grid(row=1, column=1, sticky='W', padx=10, pady=5)                                      \r\n        self.entry_db_user_name = ttk.Entry(self.tab_db, width=60, textvariable=self.conn_info.inst_login)\r\n        self.entry_db_user_name.grid(row=3, column=1, padx=10, pady=5)\r\n        self.entry_db_password = ttk.Entry(self.tab_db, width=60, textvariable=self.conn_info.inst_passwd, show="*")\r\n        self.entry_db_password.grid(row=4, column=1, padx=10, pady=(5, 10))\r\n\r\n        entry_aws_access_key = ttk.Entry(self.tab_api, width=60,\r\n                                         textvariable=self.conn_info.aws_access_key_id)\r\n        entry_aws_access_key.grid(row=1, column=1, sticky='W', padx=10, pady=(5, 5))\r\n        entry_aws_secret_key = ttk.Entry(self.tab_api, width=60,\r\n                                         textvariable=self.conn_info.aws_secret_access_key)\r\n        entry_aws_secret_key.grid(row=2, column=1, padx=5, pady=5)         \r\n        self.entry_ibm_username = ttk.Entry(self.tab_api, width=60,\r\n                                            textvariable=self.conn_info.ibm_username)\r\n        self.entry_ibm_username.grid(row=4, column=1, padx=5, pady=5)\r\n        self.entry_ibm_password = ttk.Entry(self.tab_api, width=60,\r\n                                            textvariable=self.conn_info.ibm_passwd,show="*")\r\n        self.entry_ibm_password.grid(row=5, column=1, padx=5, pady=(5,15))\r\n\r\n    def on_use_win_auth_change(self):\r\n        if (self.conn_info.use_win_auth.get() == 1):\r\n            self.entry_db_user_name.configure(state='disabled')\r\n            self.entry_db_password.configure(state='disabled')\r\n        else:\r\n            self.entry_db_user_name.configure(state='normal')\r\n            self.entry_db_password.configure(state='normal')\r\n\r\n    def on_use_ibm_api_change(self):\r\n        if (self.conn_info.use_ibm_api.get() == 0):\r\n            self.entry_ibm_username.configure(state='disabled')\r\n            self.entry_ibm_password.configure(state='disabled')\r\n        else:\r\n            self.entry_ibm_username.configure(state='normal')\r\n            self.entry_ibm_password.configure(state='normal')\r\n\r\n\r\nclass SessionDetailsFrame(ttk.LabelFrame):\r\n    def __init__(self, root, parent):\r\n        super(SessionDetailsFrame, self).__init__(root, text='2. Session Details')\r\n        super(SessionDetailsFrame, self).grid(row=1, column=0, sticky='NW', padx=5, pady=5, ipadx=5, ipady=5, rowspan=2)\r\n\r\n        self.parent = parent\r\n        self.conn_info = parent.conn_info\r\n\r\n        self.create_entries()\r\n        self.create_buttons()\r\n        self.create_scrolled_text()\r\n\r\n    def create_entries(self):\r\n        ttk.Entry(\r\n            self, justify="center", width=18, font="Helvetica 18 bold",\r\n            textvariable=self.conn_info.session_id).grid(row=1, column=2, padx=3, pady=5, sticky='W')\r\n\r\n    def create_buttons(self):\r\n        search_session_btn = ttk.Button(self, text="SEARCH SESSION ID", command=self.on_search_session_click)\r\n        search_session_btn.grid(row=1, column=3, ipadx=8, ipady=6)\r\n\r\n    def create_scrolled_text(self):\r\n        self.dialog_st = scrolledtext.ScrolledText(self, width=45, height=13, wrap=tk.WORD)\r\n        self.dialog_st.grid(column=2, row=2, padx=4, pady=4, columnspan=2, sticky='w')\r\n\r\n        style = ttk.Style()\r\n        style.configure("TButton", foreground="red")\r\n\r\n    def on_search_session_click(self):\r\n        db = MsSqlDatabase(self.conn_info)\r\n        conn = db.connect()\r\n        if conn:\r\n            results = db.get_session(conn)\r\n            if results:\r\n                self.dialog_st.delete('1.0', tk.END)\r\n                for role, message in results:\r\n                    self.dialog_st.insert(tk.END, '{}:\\n'.format(role), 'role')\r\n                    self.dialog_st.insert(tk.END, '{}\\n\\n'.format(message), 'message')\r\n                    self.dialog_st.tag_config('role', foreground='red', font="Courier 11 bold")\r\n            else:\r\n                tk.messagebox.showwarning(title="Warning", message="Nominated Session ID not found in the database!")\r\n        else:\r\n            tk.messagebox.showwarning(title="Warning", message="Cannot connect to database server!")\r\n\r\n\r\nclass PlaybackDetailsFrame(ttk.LabelFrame):\r\n    def __init__(self, root, parent):\r\n        super(PlaybackDetailsFrame, self).__init__(root, text='3. Playback Details')\r\n        super(PlaybackDetailsFrame, self).grid(row=1, column=1, sticky='WN', padx=5, pady=5, ipadx=5, ipady=5)\r\n\r\n        self.root = root\r\n        self.parent = parent\r\n        self.conn_info = parent.conn_info\r\n\r\n        self.create_labels()\r\n        self.create_combobox()\r\n        self.create_buttons()\r\n\r\n        root.protocol('WM_DELETE_WINDOW', self.on_closing)\r\n\r\n        self.process_manager = multiprocessing.Manager()\r\n        self.player_process = None\r\n        self.player_commands = None\r\n        self.player_status = None\r\n\r\n    def create_labels(self):\r\n        l1 = ttk.Label(self, text="Clinician Voice:").grid(row=0, column=0, sticky='W', padx=5, pady=5)\r\n        l2 = ttk.Label(self, text="Patient Voice:").grid(row=0, column=1, sticky='W', padx=5, pady=5)\r\n        var1 = tk.StringVar(self.root)\r\n        var2 = tk.StringVar(self.root)\r\n\r\n    def create_combobox(self):\r\n        clinician = ttk.Combobox(self, width=11, textvariable=self.conn_info.clinician_voice)\r\n        clinician.grid(row=1, column=0, padx=5, pady=5, sticky='NW')\r\n        clinician['values'] = (\r\n            'Russell',\r\n            'Nicole',\r\n            'Amy',\r\n            'Brian',\r\n            'Emma',\r\n            'Raveena',\r\n            'Ivy',\r\n            'Joanna',\r\n            'Joey',\r\n            'Justin',\r\n            'Kendra',\r\n            'Kimberly',\r\n            'Salli'\r\n        )\r\n        clinician.current(0)\r\n\r\n        patient = ttk.Combobox(self, width=11, textvariable=self.conn_info.patient_voice)\r\n        patient.grid(row=1, column=1, padx=(5, 0), pady=5, sticky='NW')\r\n        patient['values'] = (\r\n            'Nicole',\r\n            'Russell',\r\n            'Amy',\r\n            'Brian',\r\n            'Emma',\r\n            'Raveena',\r\n            'Ivy',\r\n            'Joanna',\r\n            'Joey',\r\n            'Justin',\r\n            'Kendra',\r\n            'Kimberly',\r\n            'Salli')\r\n        patient.current(0)\r\n\r\n    def create_buttons(self):\r\n        play_session_btn = ttk.Button(self, text="PLAY", width=25, command=self.on_play_session_click)\r\n        play_session_btn.grid(row=2, column=0, columnspan=2, padx=(10, 2), pady=(20, 5), sticky='WE')\r\n        pause_session_btn = ttk.Button(self, text="PAUSE", width=25, command=self.on_pause_session_click)\r\n        pause_session_btn.grid(row=3, column=0, columnspan=2, padx=(10, 2), pady=5, sticky='WE')\r\n        stop_session_btn = ttk.Button(self, text="STOP", width=25, command=self.on_stop_session_click)\r\n        stop_session_btn.grid(row=4, column=0, columnspan=2, padx=(10, 2), pady=(5, 5), sticky='WE')\r\n\r\n    def on_play_session_click(self):\r\n        if self.player_process:\r\n            if self.player_process.is_alive():\r\n                self.player_commands.put('STOP')\r\n\r\n        db = MsSqlDatabase(self.conn_info)\r\n        db_conn = db.connect()\r\n        if db_conn:\r\n            messages = db.get_messages(db_conn)\r\n            if messages:\r\n                is_credentials_valid = True\r\n                if len(self.conn_info.aws_access_key_id.get()) == 0 or \\\r\n                    len(self.conn_info.aws_secret_access_key.get()) == 0:\r\n                        is_credentials_valid = False\r\n\r\n                if (is_credentials_valid):\r\n                    credentials = {\r\n                        'aws_access_key': self.conn_info.aws_access_key_id.get(),\r\n                        'aws_secret_key': self.conn_info.aws_secret_access_key.get()\r\n                    }\r\n                    voices = {\r\n                        'clinician': self.conn_info.clinician_voice.get(),\r\n                        'client': self.conn_info.patient_voice.get()\r\n                    }\r\n                    player = AudioPlayer(credentials, voices)\r\n\r\n                    self.player_commands = self.process_manager.Queue()\r\n                    self.player_status = self.process_manager.dict()\r\n                    self.player_process = multiprocessing.Process(\r\n                        target=player.run,\r\n                        args=(messages, voices, self.player_commands, self.player_status))\r\n                    self.player_process.start()\r\n                    self.root.after(500, lambda: self.check_player_status(self.player_process, self.player_status))\r\n                else:\r\n                    tk.messagebox.showwarning(title="Warning", message="AWS access or secret key is empty")\r\n            else:\r\n                tk.messagebox.showwarning(title="Warning", message="Nominated Session ID not found in the database!")\r\n        else:\r\n            tk.messagebox.showwarning(title="Warning", message="Cannot connect to database server")\r\n\r\n    def on_pause_session_click(self):\r\n        if self.player_commands:\r\n            self.player_commands.put("PAUSE")\r\n\r\n    def on_stop_session_click(self):\r\n        if self.player_commands:\r\n            self.player_commands.put("STOP")\r\n\r\n    def on_closing(self):\r\n        if self.player_process:\r\n            if self.player_process.is_alive():\r\n                self.player_commands.put('STOP')\r\n            self.player_process.join()\r\n\r\n        self.root.destroy()\r\n\r\n    def check_player_status(self, player_process, player_status):\r\n        if not player_process.is_alive():\r\n            print('Player status: {}, {}'.format(player_status['code'], player_status['message']))\r\n            if player_status['code'] != 0:\r\n                tk.messagebox.showwarning(title="Warning", message=player_status['message'])\r\n        else:\r\n            self.root.after(500, lambda: self.check_player_status(player_process, player_status))\r\n\r\n\r\nclass WatsonGraphDetailsFrame(ttk.LabelFrame):\r\n    def __init__(self, root, parent):\r\n        super(WatsonGraphDetailsFrame, self).__init__(root, text='4. Analysis Graph Details')\r\n        super(WatsonGraphDetailsFrame, self).grid(row=2, column=1, sticky='WE', padx=5, pady=5, ipadx=5, ipady=1)\r\n\r\n        self.root = root\r\n        self.parent = parent\r\n        self.conn_info = parent.conn_info\r\n\r\n        self.create_buttons()\r\n\r\n    def create_buttons(self):\r\n        self.tone_analysis_btn = ttk.Button(self, text='PERFORM TONE ANALYSIS', width=28,\r\n                                                      command=self.tone_analysis_btn_click)\r\n        self.tone_analysis_btn.grid(row=0, column=1, padx=(12, 2), pady=(11, 11), sticky='EW')\r\n\r\n    def tone_analysis_btn_click(self):\r\n        if len(self.conn_info.ibm_username.get()) == 0 or len(self.conn_info.ibm_passwd.get()) == 0 \\\r\n            or self.conn_info.use_ibm_api.get() == 0:\r\n                tk.messagebox.showwarning(title='Warning',\r\n                                          message='\\'IBM Watson API\\' username or password is empty or disabled')\r\n                return\r\n\r\n        db = MsSqlDatabase(self.conn_info)\r\n        conn = db.connect()\r\n        if not conn:\r\n            tk.messagebox.showwarning(title='Warning', message='Cannot connect to database server!')\r\n            return\r\n\r\n        messages = db.get_messages_for_tone_analyse(conn)\r\n        if not messages:\r\n            tk.messagebox.showwarning(title='Warning', message='Nominated Session ID not found in the database!')\r\n            return\r\n\r\n        if len(messages.split()) < 3: tk.messagebox.showwarning(title='Warning', message='Too few words provided!') return if sys.getsizeof(messages) > 128000:\r\n            tk.messagebox.showwarning(title='Warning', message='The message provided is too long for API string limit.')\r\n            return\r\n\r\n        db_user_id = db.get_user_id(conn)\r\n        if not db_user_id:\r\n            tk.messagebox.showwarning(title='Warning', message='Cannot get User ID for given Session ID')\r\n            return\r\n        client = { 'session_id': self.conn_info.session_id.get(),\r\n                   'user_id': db_user_id[0][0] }\r\n\r\n        tone_analyzer = wdc.ToneAnalyzerV3(\r\n            version=self.conn_info.ibm_version,\r\n            username=self.conn_info.ibm_username.get(),\r\n            password=self.conn_info.ibm_passwd.get(),\r\n            x_watson_learning_opt_out=self.conn_info.ibm_x_watson_learning_opt_out\r\n        )\r\n\r\n        try:\r\n            tone = tone_analyzer.tone(messages, sentences=False, content_type='text\/plain')\r\n        except:\r\n            tk.messagebox.showwarning(title='Warning', message='Cannot connect to IBM Watson service')\r\n            return\r\n\r\n        dialog = ToneAnalysisDialog(self, client, tone)\r\n        self.wait_window(dialog)\r\n\r\n\r\nclass APIDetailsDialog(tk.Toplevel):\r\n    def __init__(self, parent):\r\n        super(APIDetailsDialog, self).__init__(parent)\r\n        self.parent = parent\r\n\r\n        self.title('API Details')\r\n        self.resizable(width=False, height=False)\r\n\r\n        frame = ttk.LabelFrame(self, text="Polly Text-To-Speech GUI Prototype API Details")\r\n        ttk.Label(frame, text="Text to Speech API:").grid(row=0, column=0, sticky='W')\r\n        ttk.Label(frame, text="AWS Polly").grid(row=1, column=0, sticky='W', pady=(0, 10))\r\n        ttk.Label(frame, text="Tone Analyser API:").grid(row=2, column=0, sticky='W')\r\n        ttk.Label(frame, text="IBM Watson").grid(row=3, column=0, sticky='W')\r\n        frame.pack(side=tk.TOP, fill=tk.BOTH, padx=10, pady=10)\r\n\r\n        close_btn = ttk.Button(self, text='Close', command=self.on_close_btn_click)\r\n        close_btn.pack(padx=5, pady=5, side=tk.BOTTOM)\r\n\r\n        self.update_idletasks()\r\n        w = self.winfo_width()\r\n        h = self.winfo_height()\r\n        x = (self.winfo_screenwidth() - w) \/\/ 2\r\n        y = (self.winfo_screenheight() - h) \/\/ 2\r\n        self.geometry('{}x{}+{}+{}'.format(w, h, x, y))\r\n        self.grab_set()\r\n\r\n    def on_close_btn_click(self):\r\n        self.destroy()\r\n\r\n\r\nclass ToneAnalysisDialog(tk.Toplevel):\r\n    def __init__(self, parent, client, tone):\r\n        super(ToneAnalysisDialog, self).__init__(parent)\r\n\r\n        self.parent = parent\r\n        self.client = client\r\n        self.tone = tone\r\n\r\n        self.title('Tone Analysis')\r\n\r\n        plot_widget = self.create_tone_analyse_plot()\r\n        plot_widget.pack(side=tk.TOP, fill=tk.BOTH, expand=1)\r\n\r\n        close_btn = ttk.Button(self, text='Close', command=self.on_close_btn_click)\r\n        close_btn.pack(padx=5, pady=5, side=tk.BOTTOM)\r\n        self.grab_set()\r\n\r\n    def create_tone_analyse_plot(self):\r\n        emotion_tone = {}\r\n        language_tone = {}\r\n        social_tone = {}\r\n\r\n        for cat in self.tone['document_tone']['tone_categories']:\r\n            print('Category:', cat['category_name'])\r\n            if cat['category_name'] == 'Emotion Tone':\r\n                for tone in cat['tones']:\r\n                    print('-', tone['tone_name'], tone['score'])\r\n                    emotion_tone.update({tone['tone_name']: tone['score']})\r\n            if cat['category_name'] == 'Social Tone':\r\n                for tone in cat['tones']:\r\n                    print('-', tone['tone_name'], tone['score'])\r\n                    social_tone.update({tone['tone_name']: tone['score']})\r\n            if cat['category_name'] == 'Language Tone':\r\n                for tone in cat['tones']:\r\n                    print('-', tone['tone_name'], tone['score'])\r\n                    language_tone.update({tone['tone_name']: tone['score']})\r\n\r\n        max_tone_values = list(emotion_tone.values()) + list(language_tone.values()) + list(social_tone.values())\r\n        if max(max_tone_values) > 0.9:\r\n            max_tone_value = 1\r\n        else:\r\n            max_tone_value = max(max_tone_values) + 0.1\r\n\r\n        mpl.style.use('seaborn')\r\n\r\n        fig = mpl.figure.Figure(figsize=(7, 7))\r\n        canvas = FigureCanvasTkAgg(fig, master=self)\r\n\r\n        fig.suptitle(\r\n            'Tones Analysis of Patient ID \\'{}\\', Chat Data for Session ID \\'{}\\'\\nScale range: 0 (min) -- 1 (max)'\r\n                .format(self.client['user_id'], self.client['session_id']), fontsize=14, fontweight='bold')\r\n\r\n        keys = sorted(emotion_tone.keys(), reverse=True)\r\n        values = [emotion_tone[key] for key in keys]\r\n        y_pos = np.arange(len(values))\r\n        ax1 = fig.add_subplot(311)\r\n        ax1.barh(y_pos, values, align='center', alpha=0.6, color='limegreen')\r\n        ax1.set_yticks(y_pos)\r\n        ax1.set_yticklabels(keys)\r\n        ax1.set_title('Emotion Tone', fontsize=12)\r\n        ax1.set_xlim([0, max_tone_value])\r\n\r\n        keys = sorted(social_tone.keys(), reverse=True)\r\n        values = [social_tone[key] for key in keys]\r\n        y_pos = np.arange(len(values))\r\n        ax2 = fig.add_subplot(312)\r\n        ax2.barh(y_pos, values, align='center', alpha=0.6, color='red')\r\n        ax2.set_yticks(y_pos)\r\n        ax2.set_yticklabels(keys)\r\n        ax2.set_title('Social Tone', fontsize=12)\r\n        ax2.set_xlim([0, max_tone_value])\r\n\r\n        keys = sorted(language_tone.keys(), reverse=True)\r\n        values = [language_tone[key] for key in keys]\r\n        y_pos = np.arange(len(values))\r\n        ax3 = fig.add_subplot(313)\r\n        ax3.barh(y_pos, values, height=0.4, align='center', alpha=0.6, color='deepskyblue')\r\n        ax3.set_yticks(y_pos)\r\n        ax3.set_yticklabels(keys)\r\n        ax3.set_title('Language Tone', fontsize=12)\r\n        ax3.set_xlim([0, max_tone_value])\r\n\r\n        fig.tight_layout(pad=0.9, w_pad=0.5, h_pad=1.7)\r\n        fig.subplots_adjust(top=0.85, left=0.20)\r\n\r\n        canvas.show()\r\n        widget = canvas.get_tk_widget()\r\n\r\n        return widget\r\n\r\n    def on_close_btn_click(self):\r\n        self.destroy()\r\n\r\nif __name__ == "__main__":\r\n    app = AppFrame()\r\n    app.run()\r\n<\/pre>\n
This concludes this two-part series on building a simple GUI app in Python and Tkinter using AWS and IBM machine learning cloud services. Now you can see that anyone, with a little bit of elbow grease, minimal Python skills and little bit of time to spare (no PhD required!) can take advantage of these machine learning services and create something interesting.<\/p>\n","protected":false},"excerpt":{"rendered":"
Note: Part one to this series can be found HERE In my last blog post I outlined the concept of creating a simple Python GUI application which utilised Amazon Polly Text-To-Speech cloud API. The premise was quite simple \u2013 retrieve chat data stored in SQL Server database and pass it to Polly API to convert […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[16,70,67,62,68,69,41,49,19],"class_list":["post-3024","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-analytics","tag-artificial-intelligence","tag-aws","tag-cloud-computing","tag-ibm-watson","tag-machine-learning","tag-python","tag-sql","tag-sql-server"],"aioseo_notices":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/posts\/3024","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/comments?post=3024"}],"version-history":[{"count":18,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/posts\/3024\/revisions"}],"predecessor-version":[{"id":3059,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/posts\/3024\/revisions\/3059"}],"wp:attachment":[{"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/media?parent=3024"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/categories?post=3024"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bicortex.com\/bicortex\/wp-json\/wp\/v2\/tags?post=3024"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}