added Tripartite

Author: 3rdCore

SeaPearl.html

@@ -44,7 +44,7 @@
 				<table style="width:100%;border:0px;border-spacing:0px;border-collapse:separate;margin-right:auto;margin-left:auto;">
 					<tr style="padding:0px">
 					<td style="padding:1.5%;width:16%;vertical-align:middle;text-align:center">
-						<a href="index.html" target="_self">Home</a>
+						<a href="index.html" target="_self"><big>Home</big></a>
 					</td>       
 					</tr>
 				</table>
@@ -95,7 +95,7 @@
 			<br>
 			<br>
 			<tr>
-			Now that we have introduced the notion of discrete optimisation problems, we will discuss a method used for searching solutions : <b class="term">constraint programming</b> (CP). CP is a paradigm to solve discrete optimization problems. It lies on methods for  <b class="term">problem declaration</b> and <b class="term">problem solving</b> using a solver that automates the constraint propagation mechanism. For more information about it, please look at <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjb1KrqiZX6AhU-FVkFHaUACBkQFnoECCoQAQ&url=https%3A%2F%2Fwww.cs.upc.edu%2F~erodri%2Fwebpage%2Fcps%2Ftheory%2Fcp%2Fintro%2Fslides.pdf&usg=AOvVaw3uGZ4zJqrdVZ5Z9THZWZNP" target="_blank">this presentation from Enric Rodrıguez-Carbonell </a>.
+			Now that we have introduced the notion of discrete optimisation problems, we will discuss a method used for searching solutions : <b class="term">constraint programming</b> (CP). CP is a paradigm to solve discrete optimization problems. It lies on methods for  <b class="term">problem declaration</b> and <b class="term">problem solving</b> using a solver that automates the constraint propagation mechanism (the process of inferring the possible values given the value of the variables already assigned, this is the process that everyone uses to solve a sudoku for example). For more information about it, please look at <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjb1KrqiZX6AhU-FVkFHaUACBkQFnoECCoQAQ&url=https%3A%2F%2Fwww.cs.upc.edu%2F~erodri%2Fwebpage%2Fcps%2Ftheory%2Fcp%2Fintro%2Fslides.pdf&usg=AOvVaw3uGZ4zJqrdVZ5Z9THZWZNP" target="_blank">this presentation from Enric Rodrıguez-Carbonell </a>.
 			<br>
 			the strength of the CP comes from the diversity of the problems it is able to solve. It allows, among other things, to manage non-linear constraints or variables of different types (boolean, integer ...).
 			<br>
@@ -123,13 +123,46 @@
 				<li><b class="term">Variable Selection Heuristic</b> : Select the next variable to assign</li> 
 				<li><b class="term">Value Selection Heuristic</b> : Select the value to assign to the previously selected variable</li> 
 			</ul>
-			The design of these heuristics can have a tremendous impact on search performance.
+			The design of these heuristics can have a tremendous impact on search performance, this is what makes the difference between a advanced sudoku player, who will find a solution on the first try and a beginner, who do multiple errors and changes before finding a solution. 	Moreover, for some problems, we don't even know heuristics that works well in the general case.
 			<br>
-			<br>
-			My website is still under construction. Be patient I will add the end of the explanations soon !
-        </tbody></table>
-      </td>
-    </tr>
+			This is where we bring 
+				<span class="block-line"><span><span style="color:#dd1313;">R</span><span style="color:#dd5013;">e</span><span style="color:#dd8913;">i</span><span style="color:#ddc213;">n</span><span style="color:#bedd13;">f</span><span style="color:#85dd13;">o</span><span style="color:#4cdd13;">r</span><span style="color:#13dd13;">c</span><span style="color:#13dd50;">e</span><span style="color:#13dd89;">m</span><span style="color:#13ddc2;">e</span><span style="color:#13bedd;">n</span><span style="color:#1385dd;">t&nbsp;</span></span><span><span style="color:#134cdd;">L</span><span style="color:#1313dd;">e</span><span style="color:#5013dd;">a</span><span style="color:#8913dd;">r</span><span style="color:#c213dd;">n</span><span style="color:#dd13be;">i</span><span style="color:#dd1385;">n</span><span style="color:#dd134c;">g</span></span></span> !
+				<br>
+				<br>
+    </tbody></table>
+		<table width="100%" border="0" cellspacing="15" cellpadding="10">
+			<heading><b class="term">How to learn a decision making process ?</b></heading>
+			<tr>
+				<br>
+				<br>
+				The idea is to train a model using RL to learn a smart <b class="term">Value Selection Heuristic</b> by solving successively thousands of similar problems drawn for a specific distribution and evaluate the performance of this heuristic on problems never seen during training. The resolution process still relies on CP, which guarantees the validity of the returned solutions, while the agent is in charge of branching during the search. We propose a <b class="term">generic approach</b> where a single agent could learn problems of different nature, for this purpose we present a generic graph representation of any problems using the characteristics of CP problem definition.
+				<br>
+				<br>
+				The RL algorithm used is <a href="https://www.researchgate.net/figure/The-Q-learning-algorithm-taken-from-Sutton-Barto-1998_fig1_337089781" target="_blank">Deep Q-Learning </a> (DQN). The associated <b class="term">reward</b> is based on the objective function/quality of the solution returned. 
+
+				<h3>Generic Graph representation</h3>
+				Every CP problems is defined by a set of variable, values that can be taken by these variables and a set of Constraints on these variables. The idea is to encode each of these entities as node in a graph and connect these nodes according to whether :
+				<ul>
+					<li>A <b class="term">Value</b> is part of a <b class="term">Variable</b>'s domain of definition</li> 
+					<li>A <b class="term">Variable</b> is involved in a <b class="term">Constraint</b></li> 
+				</ul>
+				Here is a little example : 
+				</tr>
+				<br>
+				<br>
+				<tr>
+					<div style="text-align: center;">
+						<img src="images/tripartite.png" alt="nthu"  width="600" height="190">
+					</div>
+				</tr>
+				<br>
+				<tr>
+					The advantage of this method is that it allows the entire information to be encoded in the graph. Each node comes with a feature vector allowing to identify -among others- the type of constraints of a Constraint node or the value of a Value node
+					<br>
+					<br>
+					My website is still under construction. I will add the end of the explanations soon  :) !	
+				</tr>
+   		 </tr>
   </table>
   <table style="width:100%;border:0px;border-spacing:0px;border-collapse:separate;margin-right:auto;margin-left:auto;"><tbody>
 	<tr>